Storage Spaces seriously slow (6-10mb write speed)

Forums › Microsoft Windows › Storage Spaces seriously slow (6-10mb write speed)

1 | 2 | 3 | 4

11390 posts

Uber Geek

Trusted

Lifetime subscriber

#2260984 19-Jun-2019 17:30

CokemonZ: The machine is multi-purpose and I have no Linux experience.
It is a
Plex server
Also use camera sync to this
Sonarr machine
Radarr machine
File server for home pcs
Qbittorrent workhorse
Backblaze backups

I am just terrified of the amount of work that will need to go into setting it up on another os that I have no experience with.

I couldn't even get ipv6 working in my edge router - showing you my level of comfort with alternative systems.

Your usecase is ideal for Unraid. You can run most of what you are doing in dockers. I was an Unraid and linux noob at the beginning of this year and went with it based on the advice here. It's excellent and there are excellent tutorials on youtube here. You almost never need to touch command line, you can do almost everything through a GUI.

I have zero regrets of going that way and it's much easier to maintain than the windows systems I had before.

CokemonZ

1051 posts

Uber Geek

#2261090 19-Jun-2019 20:21

richms:

There is a setting to say that its battery backed which when enabled will make it a crapload faster but you're at the risk of powerdowns losing your data. I turned it on for my space that I store media on since its not really a big issue when I am copying stuff onto it if it gets messed up if powered off part way thru since I can just restart it again.

Ok - now we're onto something!

So I am getting about 35 - 40mb write speed. I followed this: http://www.tecfused.com/2014/11/storage-spaces-and-parity-slow-write-speeds/

However, I am getting strange points where it stops for 3 -4 seconds and no writes at all. Do you get the same?

Large (10gb or so files) stopping in the file transfer, not at the end.

CokemonZ

1051 posts

Uber Geek

#2261136 19-Jun-2019 21:25

And it looks like the ntfs log file.
The response time for it goes through the roof, then everything dies.

michaelmurfy

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2261162 19-Jun-2019 22:32

Handle9:

Your usecase is ideal for Unraid. You can run most of what you are doing in dockers. I was an Unraid and linux noob at the beginning of this year and went with it based on the advice here. It's excellent and there are excellent tutorials on youtube here. You almost never need to touch command line, you can do almost everything through a GUI.

I have zero regrets of going that way and it's much easier to maintain than the windows systems I had before.

Fully agree - I may be a *nix systems administrator by trade but I can count on my hand the amount of times I've had to go into the command line for anything on my UnRaid server - not that I needed to, but because (for me) it was quicker than using the GUI.

Docker containers are awesome and the whole system is simple to use - it has an app store, a very active community and to be perfectly honest I'd say it is easier than Windows with setting up anything. It does boot off a flash drive so if you want to try it then just try it.

Linux is honestly far easier than most people think to use and for you it does sound better suited for your needs - using something like UnRaid is actually easier than Windows due to the well thought out web interface. It may be Linux behind the scenes but don't let this scare you off. I can think of 5 people on here who are self-confessed Linux n00bs but have learnt by either means of a Raspberry Pi or even installed UnRaid and had a good play around.

Don't be scared using an alternative operating system - UnRaid is designed to be quite easy and trust me, it beats messing around with Windows storage spaces.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

CokemonZ

1051 posts

Uber Geek

#2261244 20-Jun-2019 09:32

michaelmurfy:

Handle9:

Your usecase is ideal for Unraid. You can run most of what you are doing in dockers. I was an Unraid and linux noob at the beginning of this year and went with it based on the advice here. It's excellent and there are excellent tutorials on youtube here. You almost never need to touch command line, you can do almost everything through a GUI.

I have zero regrets of going that way and it's much easier to maintain than the windows systems I had before.

Fully agree - I may be a *nix systems administrator by trade but I can count on my hand the amount of times I've had to go into the command line for anything on my UnRaid server - not that I needed to, but because (for me) it was quicker than using the GUI.

Docker containers are awesome and the whole system is simple to use - it has an app store, a very active community and to be perfectly honest I'd say it is easier than Windows with setting up anything. It does boot off a flash drive so if you want to try it then just try it.

Linux is honestly far easier than most people think to use and for you it does sound better suited for your needs - using something like UnRaid is actually easier than Windows due to the well thought out web interface. It may be Linux behind the scenes but don't let this scare you off. I can think of 5 people on here who are self-confessed Linux n00bs but have learnt by either means of a Raspberry Pi or even installed UnRaid and had a good play around.

Don't be scared using an alternative operating system - UnRaid is designed to be quite easy and trust me, it beats messing around with Windows storage spaces.

Allright - so I've had a look and tend to agree with what you say, unraid looks easy enough - if Linus Tech Tips can pull it off, I am sure I can :)

A couple of things are appealing - it boots off USB, so while experimenting I can keep my windows install intact.

I like the idea of using a SSD as a cache. The drives I have are 2.5" and 5400 rpm so slow......so this should help. Limited to the amount of Drives I can have, so won't have mirrored caches.

So what would my migration strategy be is the question. All of my drives are NTFS formatted.

I have cleared off 3 that can be the initial array, but how would I transfer all my data from NTFS drives to unraid drives, without it flaking - linux NTFS drivers are notoriously horrible?

I also share my plex server. as this is essentially building a new one, would I need to manually add everyone back? I have plexpass if that would make things easier.

michaelmurfy

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2261300 20-Jun-2019 10:00

With Plex it is rather easy to migrate this - see: https://support.plex.tv/articles/201370363-move-an-install-to-another-system/

One of the biggest problems is you'll need to re-format your drives. As you mentioned, you do have a 10tb external you can offload to. I'd recommend re-creating your array and importing your data back. Using a SSD cache helps heaps (I have NZBGet download to the SSD cache which gets offloaded to my array every couple of hours). Anything write-intensive you can set to use your SSD cache and have it offload later to your array to speed things up.

But otherwise the beauty of doing this is any drive you have will work in the array. You need to ensure the biggest (and fastest of the biggest - would recommend a 3.5" 7200rpm spinner) drive is the Parity (or use 2x parity drives if you want to be extra safe) but otherwise you can mix and match drives a little like Storage Spaces. For example, I have 2x 8tb and 2x 3tb drives - one of the 8tb drives is parity and the rest are storage giving me 14tb of usable space. I also have a 1tb NVME cache drive.

But you won't regret it - yes, it is Linux but like we say there are a tonne of tutorials out there - check out this channel also for some tips on Plex and UnRaid:

davidcole

6034 posts

Uber Geek

Trusted

#2261302 20-Jun-2019 10:04

Following along because it's interesting, and was something I'd considered quite a bit also. How easy is it to get unraid to run a windows virtual machine? I don't actually own a computer I can sit at, I RDP from a work laptop that I have to take home. THe linux stuff I'm happy with, I run enough with VMs and odroids and raspberry pis, but I still have the odd need for windows machines (hd home run drivers, my main "developer" machine etc)

Do you run vmware on unraid? And I assume with unraid you can run a suite of docker-compose.yml files for your various services? I'd hate to have to redo it all in some proprietary gui.

Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Note that to use Quic Broadband you must be comfortable with configuring your own router.

CokemonZ

1051 posts

Uber Geek

#2261310 20-Jun-2019 10:23

michaelmurfy:

With Plex it is rather easy to migrate this - see: https://support.plex.tv/articles/201370363-move-an-install-to-another-system/

One of the biggest problems is you'll need to re-format your drives. As you mentioned, you do have a 10tb external you can offload to. I'd recommend re-creating your array and importing your data back. Using a SSD cache helps heaps (I have NZBGet download to the SSD cache which gets offloaded to my array every couple of hours). Anything write-intensive you can set to use your SSD cache and have it offload later to your array to speed things up.

But otherwise the beauty of doing this is any drive you have will work in the array. You need to ensure the biggest (and fastest of the biggest - would recommend a 3.5" 7200rpm spinner) drive is the Parity (or use 2x parity drives if you want to be extra safe) but otherwise you can mix and match drives a little like Storage Spaces. For example, I have 2x 8tb and 2x 3tb drives - one of the 8tb drives is parity and the rest are storage giving me 14tb of usable space. I also have a 1tb NVME cache drive.

But you won't regret it - yes, it is Linux but like we say there are a tonne of tutorials out there - check out this channel also for some tips on Plex and UnRaid:

I don't have enough extra storage to move everything off in one go.

So could Unraid mount my NTFS drives so I can copy stuff off, then grow the array, copy more, rinse and repeat :)

davidcole

6034 posts

Uber Geek

Trusted

#2261328 20-Jun-2019 10:29

CokemonZ:

michaelmurfy:

With Plex it is rather easy to migrate this - see: https://support.plex.tv/articles/201370363-move-an-install-to-another-system/

One of the biggest problems is you'll need to re-format your drives. As you mentioned, you do have a 10tb external you can offload to. I'd recommend re-creating your array and importing your data back. Using a SSD cache helps heaps (I have NZBGet download to the SSD cache which gets offloaded to my array every couple of hours). Anything write-intensive you can set to use your SSD cache and have it offload later to your array to speed things up.

But otherwise the beauty of doing this is any drive you have will work in the array. You need to ensure the biggest (and fastest of the biggest - would recommend a 3.5" 7200rpm spinner) drive is the Parity (or use 2x parity drives if you want to be extra safe) but otherwise you can mix and match drives a little like Storage Spaces. For example, I have 2x 8tb and 2x 3tb drives - one of the 8tb drives is parity and the rest are storage giving me 14tb of usable space. I also have a 1tb NVME cache drive.

But you won't regret it - yes, it is Linux but like we say there are a tonne of tutorials out there - check out this channel also for some tips on Plex and UnRaid:

I don't have enough extra storage to move everything off in one go.

So could Unraid mount my NTFS drives so I can copy stuff off, then grow the array, copy more, rinse and repeat :)

What format are the linux drives once they're in unraid? One of the big selling points (to me and others) with Storage spaces, drivebender, drive pool, is that the drives remain standard NTFS volumes. So if the hardware dies, you can just take a drive out and put it in any other windows machines.

If that was a consideration for while you were on windows, that might need to be confirmed.

michaelmurfy

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2261364 20-Jun-2019 11:07

davidcole:

What format are the linux drives once they're in unraid? One of the big selling points (to me and others) with Storage spaces, drivebender, drive pool, is that the drives remain standard NTFS volumes. So if the hardware dies, you can just take a drive out and put it in any other windows machines.

If that was a consideration for while you were on windows, that might need to be confirmed.

UnRaid uses XFS but like NTFS, it can be read by Windows (with a driver) or any Linux system. It can also indeed run virtual machines (and has a VM manager built in).

Copying from NTFS is a little more tricky but is possible: https://wiki.unraid.net/Copy_files_from_a_NTFS_drive

CokemonZ

1051 posts

Uber Geek

#2261387 20-Jun-2019 11:41

michaelmurfy:

davidcole:

What format are the linux drives once they're in unraid? One of the big selling points (to me and others) with Storage spaces, drivebender, drive pool, is that the drives remain standard NTFS volumes. So if the hardware dies, you can just take a drive out and put it in any other windows machines.

If that was a consideration for while you were on windows, that might need to be confirmed.

UnRaid uses XFS but like NTFS, it can be read by Windows (with a driver) or any Linux system. It can also indeed run virtual machines (and has a VM manager built in).

Copying from NTFS is a little more tricky but is possible: https://wiki.unraid.net/Copy_files_from_a_NTFS_drive

In this case - Could I create the array in unraid, boot into windows, load xfs driver, copy data, remove partitions from now empty drives, boot back into unraid, expand array, rinse and repeat?

Or I see I could create a windows VM, and I think assign the ntfs drives to it, then use that to copy...….

https://www.youtube.com/watch?v=miYUGWq6l24

michaelmurfy

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2261404 20-Jun-2019 12:07

Windows VM may be the way to go here :)

CokemonZ

1051 posts

Uber Geek

#2261470 20-Jun-2019 12:44

Unraid is $89 USD?

davidcole did you reckon there was a free version?

I see there is a 30 day trial...….

CokemonZ

1051 posts

Uber Geek

#2261484 20-Jun-2019 13:45

have any of you used www.snapraid.it It looks like it might just replace storage spaces. *ninja edit - doesn't do pooling - just parity. Would need to use drivepool or something.

I download unraid, launched it, and realised setting it up is going to be a multi day job - which I just don't have time for at the moment.

For right now I am going to stick with separate drives - as much as it is a pain in the rear.

I can't understand why the storage space just stops writing for seconds at a time though - that is super weird. And why the NTFS log file appears to be the culprit.

mentalinc

3229 posts

Uber Geek

Trusted

#2261525 20-Jun-2019 15:32

I use drivepool with snapraid.

CPU: AMD 5900x | RAM: GSKILL Trident Z Neo RGB F4-3600C16D-32GTZNC-32-GB | MB: Asus X570-E | GFX: EVGA FTW3 Ultra RTX 3080Ti| Monitor: LG 27GL850-B 2560x1440

Quic: https://account.quic.nz/refer/473833 R473833EQKIBX

1 | 2 | 3 | 4