Building a Backup Tape Machine?

Forums › Desktop computing › Building a Backup Tape Machine?

JimmyH

2886 posts

Uber Geek

#299068 8-Aug-2022 13:17

As I have posted elsewhere, I have been upgrading my setup, particularly for media streaming. With the new media server online and a new storage array, phases 1 and 2 have been completed. I'm now starting to turn my mind to proper backup, replacing the hodge-podge of USB drives I have been using.

I'm thinking of building a dedicated backup PC. Adequate older servers (ProLiant or similar) with a bunch of hard drive bays seem to go for about $450 on trademe, I can purchase 8 900GB 10K 2.5" SAS disks off ebay for about $350 landed, which should be plenty fast to feed a tape drive if I put them in a RAID array. Second hand tape drives seem pricey, but I can get an LTO5 for about $500. I would like LTO6, but they cost materially more. Given I can land tape at around $16-17/TB new (used is much cheaper), it would have significantly lower marginal cost than hard drives, and be more reliable in terms of cold storage. Plus, at that pricing I should be able to do duplicate backups and store one offsite. I currently have about 60TB of data, so it will take me a while to do that many tapes initially (I'm happy if it takes a couple of months, doing a couple whenever I have time). Since I tend to keep media, once I have dealt to that I add about 1-1.5TB a month, so it would just be writing two tapes once every 5 weeks or so.

Given the amount of data, I think the economics of tape vs hard drives stack up, even given the cost of the server. I'm not fussed about the server's power consumption because it would average being powered on for a couple of hours a week only. Say, once a week to copy that week's new data, to be backed up, to it, plus a few hours to offload it to tape every five weeks.

I'm not an IT guy so this is all new to me. I'm thinking of Linux for the server with Bacula, based on what I have read.

Is this sane, and/or am I missing anything obvious?

1 | 2

3868 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#2952085 8-Aug-2022 13:46

Before going down this route, price up cold storage (e.g. Amazon Glacier) and a utility to send/retrieve data from this cold storage.

“Don't believe anything you read on the net. Except this. Well, including this, I suppose.” Douglas Adams

Referral links to services I use, really like, and may be rewarded if you sign up:
PocketSmith for budgeting and personal finance management. A great Kiwi company.

JimmyH

2886 posts

Uber Geek

#2952089 8-Aug-2022 13:57

Not interested in a cloud provider.

Firstly, I want local backups under my control. If I go with a provider I am at their mercy regarding pricing, service changes, and random stuff (e.g, they decide to start scanning my media files and deleting ones triggering copyright filters). I prefer to run my stuff on a platform I control.

Secondly, it's not cost competitive. Glacier costs $US 0.004 per GB per month. Which might not seam like much but it's about $NZ6 per TB, or a monthly cost of $NZ 360 ($4,300 a year) odd for what I already have, which will only grow as my hoard does. A tape server would cost a fraction of this over a couple of years.

Thirdly, I know my internet package is unlimited. But uploading over 60TB would probably be pushing it, to say nothing of further backups and restores.

Fourthly, it's my hobby. I like building it and managing it so I can learn stuff.

timmmay

20589 posts

Uber Geek

Trusted

Lifetime subscriber

#2952098 8-Aug-2022 14:26

60TB is a lot for a home user. Don't forget the cost of replacing media / drives that fail. It's probably best to have two drives, so you can do a restore test on a different drive from the one that wrote the tape.

JimmyH

2886 posts

Uber Geek

#2952102 8-Aug-2022 14:45

Yep, I had figured on getting a second tape drive at some point, but the budget isn't bottomless. From what I have read, LTO is pretty reliable and has a second head in the drive that verifies as it writes. So if the write doesn't fail there is a very low probability that it's bad?

Is 60TB really a huge amount nowadays? With retail drives having hit 20TB, it's only 4 disks. Before I dumped Plex for Jellyfin I used to hang out in the Plex forums, and I also hang out on Reddit (admittedly r/datahoarder). There are folk in both those places who have materially more than ten times this much storage.

timmmay

20589 posts

Uber Geek

Trusted

Lifetime subscriber

#2952109 8-Aug-2022 14:58

60TB probably puts you in the top 0.01% of people, at a guess.

These days I guess most people have what they can fit on their phone, or in the associated cloud storage. Tech people might have a few TB of data. Pirates might have more downloaded movies and such. Without checking, I probably have 500GB of important personal data (photos and videos), and another 3TB of data that if lost I wouldn't be so worried (software installs and such).

If you have a bunch of content that can likely be sourced from the internet again I probably wouldn't bother backing it up.

gehenna

8518 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2952175 8-Aug-2022 15:53

I just upgraded my UnRAID server to 80TB with capacity for another 8 drives when I need them. PM if you want to chat, @op.

shrub

775 posts

Ultimate Geek

ID Verified

#2952212 8-Aug-2022 17:30

you say you already have a bunch of old usb drives doing your backup work. You could shuck these and use these in unraid with a few 14tb exos drives($37/tb) to top it up.

I'm a bit of a data hoarder myself and unraid has been very solid. You could also look at freenas.

JBOD software raid is the way to go over hardware raid now.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

gehenna

8518 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2952221 8-Aug-2022 18:04

Exactly what I would suggest. Get all the storage out of it's enclosures and consolidated into a system running UnRAID, at least as a starting point. Then you can just replace the oldest drives in your system as it becomes economical for you, one at a time or in bulk. E.g. I just replaced my 8TB parity disk with a 14TB one. Now I can put in any size drive up to 14TB whereas before 8TB was my limit per drive (UnRAID can't add any drives bigger than your parity drive).

UnRAID is pretty easy for a technically advanced person, touger for a mid-skilled person, and likely very difficult for a novice. I say that because even though the software itself is easy, you still need to have an understanding of the technology you want to set up before you can do it. But if what you really want to achieve is just recovering your data if a drive fails, you're better off building parity resilience in your system rather than going down the tape or offsite backup path. 1 parity disk gives you the ability to recover 1 failed drive. 2 parity disks means 2 failed drives. It depends what level of risk you are comfortable wearing. I have 1 parity disk because I know where to get the data again should I lose more than one disk.

But even the most advanced UnRAID user will likely have consumed many videos on this YouTube channel.... the guy is a legend. https://www.youtube.com/c/SpaceinvaderOne

timmmay

20589 posts

Uber Geek

Trusted

Lifetime subscriber

#2952224 8-Aug-2022 18:18

RAID is not a backup 😂 If your house burns down goodbye to all your por..... data.

wellygary

8331 posts

Uber Geek

#2952225 8-Aug-2022 18:20

JimmyH:

Yep, I had figured on getting a second tape drive at some point, but the budget isn't bottomless. From what I have read, LTO is pretty reliable and has a second head in the drive that verifies as it writes. So if the write doesn't fail there is a very low probability that it's bad?

Is 60TB really a huge amount nowadays? With retail drives having hit 20TB, it's only 4 disks. Before I dumped Plex for Jellyfin I used to hang out in the Plex forums, and I also hang out on Reddit (admittedly r/datahoarder). There are folk in both those places who have materially more than ten times this much storage.

So the Tape is purely archival?

Tape is great for cold offsite,

But You do have a Disk array as backup as well? , or is this your working copy?

It sounds like you might be short a step in not having a hot accessible backup of you working files?, maybe running some RAID mirroring/striping on the disk array...

gehenna

8518 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2952239 8-Aug-2022 19:11

timmmay: RAID is not a backup 😂 If your house burns down goodbye to all your por..... data.

Don't think I suggested it was. But since @op is saying no to off-site backup, it's moot. All depends on what is trying to be achieved. Hardware failures? RAID. Arson or some other environmental thing that takes out your location? Off-site tape storage or cloud (and RAID).

MadEngineer

4295 posts

Uber Geek

Trusted

#2952240 8-Aug-2022 19:14

Do you know how many tapes you're going to need? You said a weekly backup so your scheme should be organised as weekly then fathering to a monthly then grandfathering to a yearly. So for example every Friday do your weekly tape (5 of these) then month end do your monthly (12 tapes) then depending on how far back you want to go, at least one yearly tape (say, 3 tapes)

That's 20 tapes. Wait, you said times x2 so 40. Any other scheme = bust IMHO.

You're not on Atlantis anymore, Duncan Idaho.

JimmyH

2886 posts

Uber Geek

#2952318 9-Aug-2022 08:41

shrub:

you say you already have a bunch of old usb drives doing your backup work. You could shuck these and use these in unraid with a few 14tb exos drives($37/tb) to top it up.

No I can't. They are a real mix of drives, a considerable proportion of which (by storage) are 5TB 2.5" ones. Firstly they are SMR so wouldn't play well in Unraid. Secondly, while the Seagate ones can in theory be shucked, the WD ones have USB controllers soldered on to the board and aren't shuckable. Others are small capacity (2TB etc).

Also, I want two copies, one of which will be offsite, for long-term cold storage. Spinning drives aren't suitable for that. Tapes, with a shelf life of circa 30 years when properly stored, are.

JimmyH

2886 posts

Uber Geek

#2952319 9-Aug-2022 08:46

MadEngineer:

Do you know how many tapes you're going to need? You said a weekly backup so your scheme should be organised as weekly then fathering to a monthly then grandfathering to a yearly. So for example every Friday do your weekly tape (5 of these) then month end do your monthly (12 tapes) then depending on how far back you want to go, at least one yearly tape (say, 3 tapes)

That's 20 tapes. Wait, you said times x2 so 40. Any other scheme = bust IMHO.

Yup. I thought I spelled this out in my post. My data doesn't really change - once family videos, ripped CDs/DVDs/BDs, media captures etc get created they pretty much never change. So I want to make a complete backup of what is there, and then regular incremental backups as I add stuff. The backup will be organised as coping data to the backup machine as I add it to the main server; and then offloading it to two tape copies every time the amount of data on the backup machine reaches a tape's worth.

Tape needs are initially 80 (if I make two copies of 60TB using LTO5), or 48 if I do the same with LTO6. Once that is done I'm looking at either 24 or 14 tapes a year (LTO5 or LTO6) as I add stuff.

At a cost of around $15-17/TB for tape (landed cost if I get the tapes in 20-packs from the US, local pricing is far too high), versus around $50/TB for hard drives, aside from the better cold storage of tapes vs drives, I think the economics work of using tape rather than drives also work, even after costing the PC and tape drive needed to do it.

plas

455 posts

Ultimate Geek

#2952349 9-Aug-2022 10:24

My experience of tapes even if stored in a climate controlled vault is after a few years its down to luck if they are going to work. Maybe its better these days but I hated having to look after the backup systems.

1 | 2