Freenas Build

Forums › Self-hosted › Freenas Build

1 | 2

driller2000

935 posts

Ultimate Geek

ID Verified

#1764828 15-Apr-2017 22:49

JeremyNzl:

Update,

Just installed Unraid , due to frustrations with FreeNas, (I don't need enterprise storage)

It's awesome the plugin and apps repo is what I assume Corall was trying to achieve,

Unraid is not free but give it a look.

p.s I am to impressed with the features to start looking at the downsides of Unraid today

Have had my unRAID media server running for nearly 5 years now.

Have added multiple HDD's / swapped out smaller for larger (now @ 41TB total) / increased parity drive size / replaced a failed drive / had multiple dirty shutdowns due to power cuts (before I got a UPS) / and had to deal with the MB bios corrupting itself - all without any data loss.

So I am a massive fan and am well happy with my purchase - hope your experience is all good too.

jpoc

1043 posts

Uber Geek

#1764832 15-Apr-2017 23:34

Benoire:

So within FreeNAS, how did you do this? I will admit that my limited knowledge has obviously prevented this, hence why I've gone back to Linux and MDADM software raid (Synology DSM is built on this). I was under the impression from all the information that once you configured a single vdev that you couldn't add more drive and that you needed all drives at the same size to then expand the vdev and therefore pool?

I am doing this with FreeBSD/ZFS which opens up more of the internal configuration but I believe that I could do this with FreeNAS with a little extra effort.

I do not use raw drives, I partition my drives and give FreeBSD/ZFS the partitions. That is actually why I am using FreeBSD and not Solaris. FreeBSD is utterly glorious when it comes to partitioning large drives. Solaris sucks.

Now, for one example, suppose that I have 6 hard drives. Each of which is 6T in size. We will call the drives 0, 1, 2, 3, 4 & 5. Each drive has six partitions of 1TB. I will have 6 vdevs, a, b, c, d, e & f. Each will be raidz2 and each will have six 1TB partitions - one on each of the 6TB drives. I can suffer two drive failures and the system will still function.

Now, I buy an extra 6T drive, this is drive 6. I partition it to have 6 1TB partitions. I tell zfs to move the partition for vdev a which is on drive 0 onto drive six and I move the partition for vdev b which is on drive 1 onto drive 6. I repeat this for the partitions for vdevs b,c & d from drives 2,3 & 4 respectively and move them onto drive 6. At this point, I still have 6 vdevs, each with 6 1T partitions and no vdev has more than one partition on one physical drive. In addition, I now have six empty partitions. These partitions are all 1TB and they are on drives 0, 1, 2, 3, 4 & 6. I just create an additional vdev that uses these 6 empty partitions and I get my additional 6TB raidz2 vdev with an extra 4TB of useful data. Still, I can lose any 2 drives and still have access to all of my data.

For example 2, suppose that I have 15 2TB drives each partitioned with 4 half TB partitions. That gives me 60 partitions which will be split into 10 vdevs each of which has 6 half TB partitions spread out over those 15 disks. Now let us suppose that I replace disk 0 with a 5TB disk. I format that with 10 half TB partitions and first of all, I resilver in the four partitions that were on the original disk 0. That puts me back to where I started except that disk 0 has 10 partitions, 4 of which are used and 6 of which are empty. Of my 10 vdevs, 4 have a partition on drive 0 and 6 do not. I now select 5 vdevs that are not currently on drive 0 and from each of those vdevs I select one partition and move it onto drive 0. Of course, I make sure that the five partitions to be moved are all on different drives. At the end of this process, I now have 6 empty partitions on six different drives and I just add a new vdev with 6 half TB partitions meaning that I replaced a 2TB drive with a 5TB drive and gained an extra 2TB of useful data space with 1TB used for parity.

That is what I do. Rather than constantly add vdevs to my system, I will from time to time increase the size of the partitions that make up a vdev. My current system has hard drives of all sizes from 1.5 to 6TB and partitions of half a TB, 1TB and 2TB. I have run quarter TB and 1.5TB partitions in the past and have also used drives of half TB, 3/4 TB and 1Tb capacity. Whenever I need more storage, I just look at what drives I can get at good prices and work out how to fit them into the system and consider whether I should have an additional drive or just replace the smallest drive with a bigger one. I have been running this system for four years now and have never been in a position where I cannot increase storage capacity by just going out and buying one big disk.

It sounds complex but one you understand the basics it is just a simple logic puzzle.

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1764834 16-Apr-2017 00:03

driller2000:

Have had my unRAID media server running for nearly 5 years now.

Have added multiple HDD's / swapped out smaller for larger (now @ 41TB total) / increased parity drive size / replaced a failed drive / had multiple dirty shutdowns due to power cuts (before I got a UPS) / and had to deal with the MB bios corrupting itself - all without any data loss.

So I am a massive fan and am well happy with my purchase - hope your experience is all good too.

forgot to mention, my machine is 4 years old, so close on the same as yours.

Dirty shutdowns, apart from 12 hour rechecks, absolutely rock solid for me.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

ratsun81

508 posts

Ultimate Geek

#1764835 16-Apr-2017 00:04

jpoc:

ratsun81:

johny99: Benoire the reason for freenas is that for the price of a low end arm qnap nas I could build the equivalent of a high end i3 i5 qnap nas using freenas, it's not for a commercial use so not too worried about losing data (touch wood).

You should evaluate your requirements, if a arm based nas is deemed too expensive then why are you looking to use a product that needs expensive ECC ram for enterprise grade features that you likely dont need?

Have you looked at readynas and openmediavault? These are reputable DIY nas platforms that will remove the "requirement" for expensive ECC ram.

Sorry but this is just wrong. Actually, the phrase arrant nonsense comes to mind.

First, it is already established and understood that FreeNAS does not need ECC ram.

Second, have your looked into the pricing? Last time that I checked, ECC ram was cheaper than non ECC ram for many applications. That was last Tuesday and I looked at DDR3 pricing from amazon and aliexpress.

I understand that ECC is not required. I was only suggesting to look at what the OP wants from the storage system and work out which product will fit them best.

The whole ECC route is a minefield of if's and buts as you have already highlighted in your earlier post. Its not just about the cost of the RAM itself but the support of CPU and motherboards which adds to the cost.

When i went down the homebuilt NAS path i started looking at FreeNAS as well, after testing with the hardware i had it didnt fit my needs. NIC support is poor and the processing requirements impacted network transfer performance. I tested readynas but found it clunky and ended up using openmediavault for around 3 years. Once i got tired of spending time on updates and maintenance i bought a big brand NAS and am very happy with it.

So it always comes down to, how much is your free time worth.

Not sure the arrant nonsense comment was warranted.

Quic Broadband

Use R212389ELFLL2 promo code for free setup at checkout.

Benoire

2798 posts

Uber Geek

#1764866 16-Apr-2017 08:52

jpoc:

Benoire:

So within FreeNAS, how did you do this? I will admit that my limited knowledge has obviously prevented this, hence why I've gone back to Linux and MDADM software raid (Synology DSM is built on this). I was under the impression from all the information that once you configured a single vdev that you couldn't add more drive and that you needed all drives at the same size to then expand the vdev and therefore pool?

I am doing this with FreeBSD/ZFS which opens up more of the internal configuration but I believe that I could do this with FreeNAS with a little extra effort.

I do not use raw drives, I partition my drives and give FreeBSD/ZFS the partitions. That is actually why I am using FreeBSD and not Solaris. FreeBSD is utterly glorious when it comes to partitioning large drives. Solaris sucks.

Now, for one example, suppose that I have 6 hard drives. Each of which is 6T in size. We will call the drives 0, 1, 2, 3, 4 & 5. Each drive has six partitions of 1TB. I will have 6 vdevs, a, b, c, d, e & f. Each will be raidz2 and each will have six 1TB partitions - one on each of the 6TB drives. I can suffer two drive failures and the system will still function.

Now, I buy an extra 6T drive, this is drive 6. I partition it to have 6 1TB partitions. I tell zfs to move the partition for vdev a which is on drive 0 onto drive six and I move the partition for vdev b which is on drive 1 onto drive 6. I repeat this for the partitions for vdevs b,c & d from drives 2,3 & 4 respectively and move them onto drive 6. At this point, I still have 6 vdevs, each with 6 1T partitions and no vdev has more than one partition on one physical drive. In addition, I now have six empty partitions. These partitions are all 1TB and they are on drives 0, 1, 2, 3, 4 & 6. I just create an additional vdev that uses these 6 empty partitions and I get my additional 6TB raidz2 vdev with an extra 4TB of useful data. Still, I can lose any 2 drives and still have access to all of my data.

For example 2, suppose that I have 15 2TB drives each partitioned with 4 half TB partitions. That gives me 60 partitions which will be split into 10 vdevs each of which has 6 half TB partitions spread out over those 15 disks. Now let us suppose that I replace disk 0 with a 5TB disk. I format that with 10 half TB partitions and first of all, I resilver in the four partitions that were on the original disk 0. That puts me back to where I started except that disk 0 has 10 partitions, 4 of which are used and 6 of which are empty. Of my 10 vdevs, 4 have a partition on drive 0 and 6 do not. I now select 5 vdevs that are not currently on drive 0 and from each of those vdevs I select one partition and move it onto drive 0. Of course, I make sure that the five partitions to be moved are all on different drives. At the end of this process, I now have 6 empty partitions on six different drives and I just add a new vdev with 6 half TB partitions meaning that I replaced a 2TB drive with a 5TB drive and gained an extra 2TB of useful data space with 1TB used for parity.

That is what I do. Rather than constantly add vdevs to my system, I will from time to time increase the size of the partitions that make up a vdev. My current system has hard drives of all sizes from 1.5 to 6TB and partitions of half a TB, 1TB and 2TB. I have run quarter TB and 1.5TB partitions in the past and have also used drives of half TB, 3/4 TB and 1Tb capacity. Whenever I need more storage, I just look at what drives I can get at good prices and work out how to fit them into the system and consider whether I should have an additional drive or just replace the smallest drive with a bigger one. I have been running this system for four years now and have never been in a position where I cannot increase storage capacity by just going out and buying one big disk.

It sounds complex but one you understand the basics it is just a simple logic puzzle.

No, not really.. you're effectively building hybrid raidZ2s with multiple vdevs. Hybrid raid is generally partitioned drives as you've suggested, although it is trickier with ZFS. My point does still stand that FreeNAS does not support this in the gui and you'll get lambasted in the forums if you do this and you're at cli only in the 'appliance' to achieve this.

Are you resilvering on each partition move? This would add more stress to all the other drives as they're rebuilt from the pool?

Benoire

2798 posts

Uber Geek

#1764868 16-Apr-2017 08:55

ratsun81:

jpoc:

ratsun81:

johny99: Benoire the reason for freenas is that for the price of a low end arm qnap nas I could build the equivalent of a high end i3 i5 qnap nas using freenas, it's not for a commercial use so not too worried about losing data (touch wood).

You should evaluate your requirements, if a arm based nas is deemed too expensive then why are you looking to use a product that needs expensive ECC ram for enterprise grade features that you likely dont need?

Have you looked at readynas and openmediavault? These are reputable DIY nas platforms that will remove the "requirement" for expensive ECC ram.

Sorry but this is just wrong. Actually, the phrase arrant nonsense comes to mind.

First, it is already established and understood that FreeNAS does not need ECC ram.

Second, have your looked into the pricing? Last time that I checked, ECC ram was cheaper than non ECC ram for many applications. That was last Tuesday and I looked at DDR3 pricing from amazon and aliexpress.

I understand that ECC is not required. I was only suggesting to look at what the OP wants from the storage system and work out which product will fit them best.

The whole ECC route is a minefield of if's and buts as you have already highlighted in your earlier post. Its not just about the cost of the RAM itself but the support of CPU and motherboards which adds to the cost.

When i went down the homebuilt NAS path i started looking at FreeNAS as well, after testing with the hardware i had it didnt fit my needs. NIC support is poor and the processing requirements impacted network transfer performance. I tested readynas but found it clunky and ended up using openmediavault for around 3 years. Once i got tired of spending time on updates and maintenance i bought a big brand NAS and am very happy with it.

So it always comes down to, how much is your free time worth.

Not sure the arrant nonsense comment was warranted.

For the cost of a cheap Supermicro X8 motherboard with a L5630 & DDR3 Rdimm (with ECC) you'll get a decent performing, relatively low power (my 32GB single xeon on X8DTE-F with 16 bay SM chassis) that consumes around 70w. The motherboard, memory and cpu (and if you look for a X8DT6 with inbuilt LSI2008 controller) will cost around $300 or so from ebay.

Mantis

96 posts

Master Geek

ID Verified

#1766864 18-Apr-2017 16:28

I've been running Freenas for few years now I find it rock solid. It probably runs better on server hardware with ECC memory but there are plenty of people running it on all sorts of systems.

I'm using a Supermicro X9 motherboard with ECC memory and a some Pentium CPU. It was a bit more expensive than desktop hardware, but was worth it for the IMPI remote management alone, don't think I've ever had a keyboard/monitor connected, you can do everything including installing an OS and changing BIOS settings remotely through a web browser. The motherboard also has 2 intel NIC's built in which are well supported in Freenas and would recommend a Supermicro or similar for any home built NAS.

There is bit of a learning curve setting up Freenas but it's pretty simple once you get the hang of it. I've had a disk fail and rebuilding was not trouble and didn't lose any data.

I'd recommend for any NAS:
-Supermicro motherboard-any modern dual core CPU is fine, maybe quadcore/Xeon if you'll be running VM's/docker containers/Plex transcoding etc.
-as much RAM as you can afford, it's pretty cheap and ZFS will be able to use it. ECC is recommended (but not required) if your system supports it
-a good quality power supply (Seasonic is recommended) and a case that keeps you drives cool
-understand how ZFS works (if you're using it) and how it can be expanded (not easily)
-Intel network cards for Freenas
-if you want more disks, there are good PCIe SATA/SAS cards (IBM M1015 or similar) for about $100 for another 8 SATA ports

I guess when I built my NAS it cost a similar amount to a basic Synology/Qnap etc (without disks) but I liked having the flexibility to change OS, or have more than 4/6/8 disks and I enjoyed building it.

Sharesies kids

Free kids accounts - trade shares and funds (NZ, US) with Sharesies (affiliate link).

gzcwnk

23 posts

Geek

#1771778 28-Apr-2017 11:46

No you dont need ECC.

I have tried a few different NAS ideas (freenas incl) over the years and I have settled on gluster. So I have 3 Dell 900s I got for $100 each with a 1TB disk in each and have a 3 way Raid1 setup. It primarily acts as the NAS storage array for my VMware server over NFS amongst other things and I beat the crap out of it as a result and it doesnt miss a beat. Write disk i/o however isnt as good as I'd like but then I have triple redundancy for my data, so its a tradeoff I made.

darylblake

1162 posts

Uber Geek

Trusted

#1771821 28-Apr-2017 12:31

Was interested in doing this too. It got too expensive.

I looked at buying a NAS. But could spend close to the same amount of money on drives. Pop them in a PC i already had an raid it, and for my purposes it was ok.

You need a decent amount of ram if you have a lot of disks. Sure ECC would probably be better. But at the end of the day it depends how much you want to spend.

1 | 2