Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


View this topic in a long page with up to 500 replies per page Create new topic
1 | 2 | 3
toyonut
1508 posts

Uber Geek


  #1202889 23-Dec-2014 20:23
Send private message

starting at $1000 ish without disks or much RAM. 




Try Vultr using this link and get us both some credit:

 

http://www.vultr.com/?ref=7033587-3B


 
 
 

Free kids accounts - trade shares and funds (NZ, US) with Sharesies (affiliate link).
jpoc
1043 posts

Uber Geek


  #1203970 26-Dec-2014 11:25
Send private message

I use Freebsd and ZFS. I have done so for years and have never lost any data despite hardware failures, disk failures and power outages.

It is easy to setup and run - I export shares with samba.

The hardware requirements are low. Ignore all the exaggerated horror tales about gigs of RAM per terabyte of disk. I have an old core 2 duo running with 4G of memory and it is just fine. You can also ignore all the guff you get from the likes of PB tech who will try to upsell you from a Green drive to a Red one. ZFS works well on all drive types.

BTRFS is claimed to do all that ZFS can do and more but every time I looked, important features are still in beta. ZFS has been heavy duty industrial grade production ready for years.

All you would need to get going would be a cheap old desktop from trademe and the disks that you want to use. I ssh in to my server so no need for another monitor.

I chose freebsd rather than solaris because freebsd does gpt partitioning really well. You can give each gpt partition an arbitrary label and then access partitions for ZFS with the labels and you can forget about the weird and wacky /dev/sda5p3 stuff. That is also an argument for using Freebsd and not Linux for this job.

Be aware that windows can quietly lose data as it writes to a file. I tested this out with fastsum and a binfile-diff utility a few years back. Windows would happily copy a large file from one disk to another with no reported errors. Fsum would report different md5 sums and the bin diff would highlight that the destination file was the same length as the original but that the dest file contained a block of some small number of Kbytes that were set to zero! I no longer use windows to copy any files. I use either teracopy (free edition) or, over bad networks, I use a program that I wrote that will cope with network errors and outages. Tera copy does its own checksums and if I use my program, I then use fsum. I have seen these windows errors on w2k, XP and 7. I daresay that vista and 8 will be no better.

For backups, I do this:

Split the files to be backed up into four lots of equal size.

Create a rar file for each of those four lots. The rar file will be a multi volume one and it will have a small number of recovery volumes.

I will then use quickpar over all of the resultant files to generate 50% redundant recovery files.

I write all of the rar files for the first set onto disk 1 and and so on up to disk four and then I write the par2 files onto disks 5 and 6.

I can then stash the disks into 6 different offsite locations.

If I want to recover one file, I just need to fetch the one of those six disks onto which the original file was rar'ed from its location. If the disk has bad sectors, then the rar recovery volumes will deal with that. If the whole disk has failed then I need to get at least four of the remaining 5 disks and quickpar will restore a copy of the failed disk and I can extract the file that way.



jpoc
1043 posts

Uber Geek


  #1203992 26-Dec-2014 12:18
Send private message

Mark: Running MD5 on top of your existing filesystem is a waste of effort to be honest, if MD5 detects the error it's too late, you need those checks done by the filesystem or underneath it.

It's amazing how many enterprise environments don't even think of bit rot and just assume the storage arrays handles it, RAID does it right ?  Oh noes it don't! :-(

So running a good modern filesystem is a must if you want to make sure your TiB of data is still the same TiB of data from 6 months ago, ZFS, BTRFS and ReFS all have the capability to keep checking and rechecking, but you need to make sure the machine itself is up to spec as well (so a UPS and ECC RAM are essential options), and there is the side effect that the discs will be active nearly all the time adding up the power bill, cooling requirements and noise.



ZFS has no need of a UPS. Raid 5 has a bug - the raid 5 write hole - which means that UPS is a must for raid 5 or raid 6 if you want to avoid data loss after a power outage. ZFS was written specifically to not have that weakness.




Mark
1623 posts

Uber Geek


  #1204035 26-Dec-2014 13:53
Send private message

jpoc: 

ZFS has no need of a UPS. Raid 5 has a bug - the raid 5 write hole - which means that UPS is a must for raid 5 or raid 6 if you want to avoid data loss after a power outage. ZFS was written specifically to not have that weakness.



I don't agree about not needing a UPS ... data going into the server does not instantly get written to the ZIL it's in RAM for an eternity (to a computer) so having a UPS hooked up gives the server time to do a controlled destine/shutdown.

ZFS is good, but it's not the holy saviour of data storage :-)  It's more than capable for home use but you do have to make sure it has its essential tools (UPS, ECC RAM etc)  and it is resource hungry (which is why not too many consumer NAS implementations use it ... that and licensing probably) If you can't meet it's minimum requirements  you may as well just build a simpler RAID-5 or RAID-6 setup.


timmmay

19757 posts

Uber Geek

Trusted
Lifetime subscriber

  #1204048 26-Dec-2014 14:21
Send private message

The Mrs has agreed data integrity is a priority. Any more nas recommendations that use a good robust reliable file system? Price is significant, prefer low maintenance, device rather than pc most likely. Min 2 disks, maybe 4 would be nice.

timmmay

19757 posts

Uber Geek

Trusted
Lifetime subscriber

  #1204049 26-Dec-2014 14:22
Send private message

And thanks to jpoc for the huge useful interesting post :) Sounds good but prefer simple, ie nas.

jpoc
1043 posts

Uber Geek


  #1204054 26-Dec-2014 14:37
Send private message

timmmay: And thanks to jpoc for the huge useful interesting post :) Sounds good but prefer simple, ie nas.


Then use FreeNAS - it is a FreeBSD/ZFS distro that is pre-configured to be a consumer NAS. It is well supported by community forums.



timmmay

19757 posts

Uber Geek

Trusted
Lifetime subscriber

  #1204090 26-Dec-2014 16:23
Send private message

Sounds good but I don't want a big pc, something tiny and fit for purpose maybe easiest for me. I know you can get small pc's but a dedicated device maybe easier.

SaltyNZ
7367 posts

Uber Geek

Trusted
2degrees
Lifetime subscriber

  #1204111 26-Dec-2014 17:01
Send private message

timmmay: Sounds good but I don't want a big pc, something tiny and fit for purpose maybe easiest for me. I know you can get small pc's but a dedicated device maybe easier.


I'm pretty happy with the ReadyNAS but I'm a little suspicious of both disks in it failing within a month of each other. That might be a bad batch (of disks), maybe? But it might be heat. The system seems to run relatively hot.




iPad Pro 11" + iPhone 15 Pro Max + 2degrees 4tw!

 

These comments are my own and do not represent the opinions of 2degrees.


jpoc
1043 posts

Uber Geek


  #1204115 26-Dec-2014 17:09
Send private message

SaltyNZ:
timmmay: Sounds good but I don't want a big pc, something tiny and fit for purpose maybe easiest for me. I know you can get small pc's but a dedicated device maybe easier.


I'm pretty happy with the ReadyNAS but I'm a little suspicious of both disks in it failing within a month of each other. That might be a bad batch (of disks), maybe? But it might be heat. The system seems to run relatively hot.


Yes, heat is a big issue in these things. small plastic cases with inadequate fans......

I have a couple of 6T drives and they run hot even in a big well vented case.


jpoc
1043 posts

Uber Geek


  #1204134 26-Dec-2014 17:36
Send private message

timmmay: Sounds good but I don't want a big pc, something tiny and fit for purpose maybe easiest for me. I know you can get small pc's but a dedicated device maybe easier.


In your case, I might go for something like this off trademe.

http://www.trademe.co.nz/computers/desktops/no-monitor/auction-825070257.htm

I believe that you can fit three drives in there - a boot drive and two data drives - and it will go up to 8G of RAM.

FreeNAS does not need an EFI BIOS to run with 6T disks.

Sure that is an old machine but I have never actually had an ancient Dell desktop fail on me. (Same for IBM/Lenovo.)



richms
26749 posts

Uber Geek

Trusted
Subscriber

  #1204145 26-Dec-2014 18:00
Send private message

Old machines will use more power than a new machine doing the same things so those budget ex lease machines are often a false economy for always on applications.




Richard rich.ms

timmmay

19757 posts

Uber Geek

Trusted
Lifetime subscriber

  #1206151 30-Dec-2014 16:58
Send private message

Based on the advice from this thread I've started a thread to ask for advice about a NAS - if anyone can contribute the thread's here :)

gbwelly
1209 posts

Uber Geek


  #1206158 30-Dec-2014 17:34
Send private message

Before you spend any money have you tried calculating the checksum using an alternative tool to verify it's not a bug in fastsum?








timmmay

19757 posts

Uber Geek

Trusted
Lifetime subscriber

  #1206160 30-Dec-2014 17:41
Send private message

May do that thanks.

1 | 2 | 3
View this topic in a long page with up to 500 replies per page Create new topic





News and reviews »

One New Zealand Extends 3G Switch-off Date
Posted 11-Apr-2024 08:56


Amazon Echo Hub Review
Posted 10-Apr-2024 18:57


Epson Launches New Versatile A4 Desktop Scanners
Posted 10-Apr-2024 15:31


Motorola Mobility Launches New Android Phones in New Zealand
Posted 10-Apr-2024 14:59


Logitech G Unveils the PRO X 60 Gaming Keyboard
Posted 9-Apr-2024 19:01


Logitech Unveils Signature Slim Keyboard and Combo
Posted 9-Apr-2024 13:33


ExpressVPN Launches Aircove Go Portable Router With Built-in VPN
Posted 26-Mar-2024 21:25


Shure MoveMic Review
Posted 25-Mar-2024 12:47


reMarkable 2 Launches at JB Hi-Fi New Zealand
Posted 20-Mar-2024 08:36


Samsung Galaxy S24 Ultra review
Posted 19-Mar-2024 11:37


Google Nest Wifi Pro Review
Posted 16-Mar-2024 11:28


Samsung Galaxy A55 5G and Galaxy A35 5G
Posted 12-Mar-2024 12:41


Cricut EasyPress Mini Zen Blue launches at Spotlight New Zealand
Posted 12-Mar-2024 12:32


Logitech Introduces MX Brio Webcam
Posted 12-Mar-2024 12:24


HP Unveils Broadest Consumer Portfolio of AI-Enhanced Laptops
Posted 3-Mar-2024 18:09









Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.







GoodSync is the easiest file sync and backup for Windows and Mac