Home server storage - SSDs, RAID, etc

Forums › Self-hosted › Home server storage - SSDs, RAID, etc

chevrolux

4962 posts

Uber Geek
Inactive user

#251162 11-Jun-2019 13:29

So my home server runs on average consumer hardware... i5 6400, 32GB RAM, cant remember motherboard model, Asus something, six SATA's onbaord with RAID available.

I run Proxmox for the Hypervisor, that won't be changing. The hypervisor and the virtual disks are on a pair of NVMe drives in RAID 1 - ultra-fast no issues with that.

I then have 4x WD Red drives that I have had since the beginning of time apparently - clocking up 48000+ hours each. They are in a ZFS 'raid 0' with a 128GB SSD cache drive - built within Proxmox. And then I have a FreeNAS VM where that array is mounted.

Problem is read/writes can be just abysmal and with the introduction of a 4K HDR TV, movie play back can suck sometimes unless its the only thing trying to do stuff to that disk array. The biggest issue is the massive variances in speeds I see - I assume when the SSD caches gets full the performance plummets. As reported by Windows, reads hover around 25MB/s but writes can plummet way down to 10-15MB/s.

The server has dual gigabit NIC's on a 802.3ad LAG.

So the two options I see are:

1) SSD's

4x Samsung 860 2TB drives - $401ea - $1604 (and use onboard SATA and onboard RAID, to do a RAID 5)

+ Hardware RAID card to reconnect the old 2TB drives for stuff that can be slow (photos, documents etc)

2) New HDD's & Hardware RAID

4x Ironwolf Pro's (they seem quite cheap for 'pro') 2TB - $221ea - $884

+ Hardware RAID card and do a RAID 5 set up?

At this point, I intend on ditching FreeNAS and probably go to OpenMediaVault as I don't think I get any benefit from ZFS and I can use a linux container rather than VM to spare up some RAM.

The NAS storage is really just for movies and TV, and some photo storage.

What would you do?

Considerations:
- I don't really want to spend the money on SSD's
- I don't care about redundancy - tv and movies aren't important. I know I'm talking about RAID 5, but the final goal is just better performance - if I achieve that without RAID, so be it.
- Will also probably stick a 10Gig network card in as I have SFP+'s on my switch.

hio77

'That VDSL Cat'
12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

Subscriber

#2256065 11-Jun-2019 13:56

i run an unraid box, Love it.

it does the full 1gbit for reads, it does the parity.

Writes hover at the 30-60MB/s mark with solid 1Gbit until ram cache fulls (it's an old Athlon X2, it's CPU limited not disk limited)

I'd really stick to software raid for your usecase.

The biggest thing stopping me from nas'ing all my storage, is all the tier storage solutions currently are only Block level for windows.

I'd love to stick a 2TB SSD or two, infront of a network storage, have it cache what i'm using and move everything off to spinning disks in the network.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

Tradepub

Free professional, reference and technical white papers (affiliate link).

gehenna

8452 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2256078 11-Jun-2019 14:21

+1 Unraid here. It's bloody awesome. 40TB array with parity, 1TB SSD cache, 500GB SSD for VMs. Docker containers, plugins, VMs, etc. It's fantastic.

chevrolux

4962 posts

Uber Geek
Inactive user

#2256091 11-Jun-2019 14:35

Would unraid's software raid give me a decent performance boost though compared with just a hw raid card and virtual disk?

I'm not too keen to move away from Proxmox for virtualisation. All the VMs (or containers to be accurate) are set up and working perfectly, it's just the spinning rust that I think is my issue.

hio77

'That VDSL Cat'
12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

Subscriber

#2256094 11-Jun-2019 14:38

chevrolux: Would unraid's software raid give me a decent performance boost though compared with just a hw raid card and virtual disk?

I'm not too keen to move away from Proxmox for virtualisation. All the VMs (or containers to be accurate) are set up and working perfectly, it's just the spinning rust that I think is my issue.

No it will give you a drop unless you move to the new fangled RAID options in unraid.

Personally i still use ResizerFS, Every disk is able to be removed and read manually should the worst happen. Perfect for Content Archives.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

danielparker

222 posts

Master Geek

Trusted

Lifetime subscriber

#2256095 11-Jun-2019 14:39

I have successfully run Proxmox in a KVM on Unraid for running my existing LXC's.. It was just another node in my cluster as far as Proxmox was concerned.

Daniel

Mark

1653 posts

Uber Geek

#2256098 11-Jun-2019 14:44

Possibly you would get better performance by passing the WD harddisks through to the FreeNAS VM, if I read your config right you have your Proxmox host configuring the disks as RAID-0 ZFS (aweful idea by the way), then you are running a FreeNAS VM which in turn will have it's own virtual disks on the hosts ZFS filesystem and it then puts filesystems on top (probably ZFS again ?) .. that would suck for performance.

To go you a performance comparison ... I had an old Linux based NAS with an Atom D525 and 3GiB RAM with some equally elderly 2TB drive in software RAID-5 (MDM) with LVM on top of that and a big EXT4 filesystem. It can max out 2 * 1Gbit links in LACP trunk for both reads and writes sustained.

Personally I wouldn't do hardware RAID, it limits portability (moving the disks and data to another system) and data recovery options and doesn;t really give much more in performance for what you want.

chevrolux

4962 posts

Uber Geek
Inactive user

#2256193 11-Jun-2019 16:26

Ok so good points re expanding hardware raid.

@hio77 you mention saturating 1Gb link. Is that with spinning disks? Consumer 7200s?

And if I move to unraid, how does that parity drive work in terms of sizing? I need around 6-8TB usable storage.

hio77

'That VDSL Cat'
12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

Subscriber

#2256362 11-Jun-2019 21:12

chevrolux: Ok so good points re expanding hardware raid.

@hio77 you mention saturating 1Gb link. Is that with spinning disks? Consumer 7200s?

And if I move to unraid, how does that parity drive work in terms of sizing? I need around 6-8TB usable storage.

Yes, i do buy only toshibas though, they are actually quite well priced and insanely fast.

Their only downside is there are loud and do produce a fair bit more heat than a standard disk. - parity needs to be the largest or equal largest disk.

This is a straight copy from box to one of my SSD's, it's a little sluggish right now as there is content streaming but hitting 100MB/s is pretty nice for me.

For only 8TB, your really more in the market for a standard nas off the shelf tbh. that can be done in 2 Disks...

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

chevrolux

4962 posts

Uber Geek
Inactive user

#2256379 11-Jun-2019 21:30

That's awesome thanks. I've been in half a mind to just buy a NAS and just leave this machine to do all the VM's. But then I kind of like it all being in one box - it's also been good to be able to have virtual NICs to mount the NAS shares on the VM's.

Turns out I can get those Ironwolf Pro 2TB's for $147 each, not sure I will beat that and because they're cheap will grab six of them to fill up the SATA ports. Will take a look at the toshiba drives though.