The Great Backup Conundrum

Forums › Desktop computing › The Great Backup Conundrum

deepred

497 posts

Ultimate Geek

ID Verified

Trusted

#66824 24-Aug-2010 12:42

I'm planning to backup my data, and I've run into a conundrum.

DVD's - good for archival, but given the amount of data I plan to backup (approaching a few terabytes at least), I've outgrown even dual-layer discs.

External HDD's - cost effective, but from past experience, not the most reliable medium.

Bluray discs - big enough storage and good for archival, but burners are still horribly expensive.

HD-DVD - may be cost effective, but Toshiba pulled the plug on it, and availability and warranty are going to be issues.

Tape/QIC - even more expensive than Bluray, and normally used only in corporate environments.

Solid state storage - quick & somewhat reliable, but still quite expensive by the gigabyte.

Every backup method, it seems, has a fishhook. Any other ideas? I've been working in the computer parts biz for a while now, and I can't believe I'm asking these questions.

"I regret to say that we of the F.B.I. are powerless to act in cases of oral-genital intimacy, unless it has in some way obstructed interstate commerce." — J. Edgar Hoover

"Create a society that values material things above all else. Strip it of industry. Raise taxes for the poor and reduce them for the rich and for corporations. Prop up failed financial institutions with public money. Ask for more tax, while vastly reducing public services. Put adverts everywhere, regardless of people's ability to afford the things they advertise. Allow the cost of food and housing to eclipse people's ability to pay for them. Light blue touch paper." — Andrew Maxwell

nickd

447 posts

Ultimate Geek

#372055 24-Aug-2010 12:47

What about an online solution? I currently use Mozy. But I don't have as much to back up as you do....

xpd

Geek @ Coastguard NZ
13769 posts

Uber Geek

Retired Mod

ID Verified

Trusted

Lifetime subscriber

#372057 24-Aug-2010 12:49

If your data is that important to you, then cost should not be an issue really.....

Gavin / xpd / FastRaccoon / Geek of Coastguard New Zealand

LinkTree

timbosan

2159 posts

Uber Geek

#372067 24-Aug-2010 13:04

xpd: If your data is that important to you, then cost should not be an issue really.....

Oh so true. It's not the cost of replacing hardware or re-installing operating systems or applications, its the digital memories that once gone are lsot forever, you can never place a price on them. I have been lucky and not lost anything except for few old emails (knocks on imitation wooden desk).

I too suggest online backups - I have used Carbonite very well, in conjunction with a local server that backs up all workstations to multiple drives for redundancy (a mixture of OSX and Windows clients). You can never have too many backups!

graemeh

2078 posts

Uber Geek

#372075 24-Aug-2010 13:13

timbosan: I too suggest online backups - I have used Carbonite very well, in conjunction with a local server that backs up all workstations to multiple drives for redundancy (a mixture of OSX and Windows clients). You can never have too many backups!

Slightly off topic but how do you do the workstation backups to the local server?

davidcole

6037 posts

Uber Geek

Trusted

#372079 24-Aug-2010 13:15

Windows Home Server does local backups of all windows machines. Then couple that with carbonite for clound back up as well for certain things (photos mostly).

Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight

timbosan

2159 posts

Uber Geek

#372109 24-Aug-2010 13:58

graemeh:
timbosan: I too suggest online backups - I have used Carbonite very well, in conjunction with a local server that backs up all workstations to multiple drives for redundancy (a mixture of OSX and Windows clients). You can never have too many backups!

Slightly off topic but how do you do the workstation backups to the local server?

I use Windows Home Server (the same as 'davidcole' mentions). It supports Windows clients through a piece of software installed on the client, which performs a backup overnight, and allow restoring from the last 10 or so backups.

One of the more powerful features is that you can do a 'bare metal' restore - that is from a CD you can boot and select the workstation and backup image to restore and the whole machine will be restored, O/S, apps, everything. This is great and I used it once when I replaced a hard drive in my laptop - did a backup, swapped the hard drive - booted of the CD - choose the image and 1 hour latter I had everything the same as that morning.

Offically though WHS doesn't support OSX client (well actually, OSX doesn't support backing up to WHS), there are easy ways around this and it means Time Machine (the OSX backup software) maintains a constant backup. I think it's hourly, and keep the last 24 hours, then the last 7 days, then the last 4 weeks etc etc.

AFAIK you can download/order a DVD with a 120 day demo of WHS (I have one somewhere, if I can find it and your in Auckland you can use it - I assume this is withing geekzone policy as it is only a demo and licensed as such, moderators?).

graemeh

2078 posts

Uber Geek

#372118 24-Aug-2010 14:08

Thanks davidcole and timbosan, I might have to investigate WHS although at the moment I'm trying to avoid adding yet another box to the network :)

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

Talkiet

4793 posts

Uber Geek

Trusted

#372122 24-Aug-2010 14:12

deepred: I'm planning to backup my data, [snip]

GOOD MAN!

A) You want to backup your data
B) You've asked for advice when you ran into problems.

You're light years ahead of most people already.

I have a similar situation - I have a LOT of photos (RAW and JPG) as an amateur sports photographer to backup - currently about a terabyte. It's scary...

Here's what I have.

All photos (and my personal data as well) are on...

- My main PC on their own drive which I replace every 12 months
- Server in garage with proper RAID system (EONSTOR) running RAID5, synced daily
- Two sets of hard drives used in an eSATA caddy - one set stored in my desk drawer at work, the other at home and updated/switched every month or after a major event.
- A full backup through Backblaze (www.backblaze.com). The initial sync took about 4 months (on bigtime) but now it's just maintenance so I can deal with the bandwidth.

In your case, I'd advise 2 sets of backup HDDs, store one set offsite and rotate them every few weeks. Anything else for that amount of data is just prohibitively expensive or awkward. A RAID system isn't a backup if it's your only storage.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

deepred

497 posts

Ultimate Geek

ID Verified

Trusted

#392868 17-Oct-2010 17:07

Thanks for all the advice. The Great Backup Conundrum has been solved... I think.

It involves the use of internal HDDs - but with mobile racks. I'm still kind of suspicious about the reliability of USB caddies.

The product below is the one I used:

iSwap 201 SATA Hot Swap mobile rack

freitasm

BDFL - Memuneh
79309 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#392869 17-Oct-2010 17:15

I use a Windows Home Server at home, plus Mozy Home - currently with 130 GB copied online.

ioquatix

50 posts

Geek

#395461 25-Oct-2010 12:42

I recently put together some tips about the different things to consider when putting together a backup policy.
http://www.drobo.co.nz/blog/2010-10/backup-policy

I hope this helps.

With respect to your question about the various kinds of media available.. ultimately having your data in more than one place will ensure that after a data loss event you have the maximum chance of recovering said data. I normally recommend having offsite RAID storage and having incremental backups to at least one offsite location.

As mentioned in the article, there are many different providers for backup, online backup is certainly one area which can reduce the complexity and hardware costs of any kind of backup solution.

Kind regards,
Samuel

meesham

973 posts

Ultimate Geek

#396066 26-Oct-2010 23:32

I have a few different ways methods in my backup solution. My network is as follows:

- A linux box with a 6TB RAID-6 array and a 4TB RAID-5 array.
- 3 Windows 7 desktops.
- 2 Windows 7 laptops

I run GFI Home Backup on each of the workstations (it's free and supports VSS) that does a sync to a "Backup" windows share on the Linux box 4TB array. The backup runs once a night on the desktops and on demand on the laptops, creating a synced copy of the files on the server.

On the server an rsync process then runs a couple of hours later that copies the super-important files (mainly photos) to the 6TB array, so there's always 2 copies of the photos on the server.

About 15 minutes later an rsync process is run again that copies the entire contents of the Backup share to an encrypted 2TB USB3 hard disk (one of the new Samsung "Story" drives, very quick!). This is basically my run-out-the-door solution, it's always on and connected so if there's a fire or it gets stolen it's useless, but if we have to leave the house quickly (like the recent earthquake we had here in Christchurch) I don't have to worry about grabbing the whole server (it has a lot of hard drives in it and isn't light!). I plan to get a second encrypted 2TB drive that will live at a friend's place about 1km away and swap them every couple of weeks.

My final backup method is using an online backup service, namely Crashplan. I tried out quite a few different services (Mozy, Carbonite, Jungledisk, Memopal etc etc) but these guys seem to have the nicest solution for the price (I also really liked Jungledisk but found it a bit too expensive). I've paid extra for Crashplan+ which gives realtime backups (so it's constantly watching for changed files) and a few other cool features. The free version supports peer-to-peer backups with your friends which is encrypted so they can't see them, you can also seed it from a backup to an external hard drive to save on bandwidth costs. The Linux client works quite nicely and I'm uploading at 1Mbps on Telstra cable, not full speed but good enough for me. Only certain directories from the workstations are backed up (basically the "My Documents" folders) and all the photos to save my monthly quota.

As you can probably tell the photos are the most important thing to us which is why there's 5/6 copies of them floating around. I have a couple of TB of recordings/movies/tv shows and music, but I've decided that it's not worth backing them up - they live on the 6TB array and while there's redundancy in the storage if the server gets stolen/blows up etc I lose it all, but I can live with that.

ioquatix

50 posts

Geek

#396069 26-Oct-2010 23:47

One thing to keep in mind about a solution that relies on a single backup server is the potential for power spikes and surges to destroy entire racks of equipment.

Having data on two sets of disks attached to the same computer is great, but if you have a power surge you run the risk of loosing everything.

One way to mitigate this risk is to have a disconnected "offline" backup solution.

Using a UPS is a good idea but even those aren't completely fail safe from my experience.

Kind regards,
Samuel