Musings on rolling your own remote "cloud" backups and general backups

Forums › IT Pro and developers › Musings on rolling your own remote "cloud" backups and general backups

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784216 18-May-2017 13:44

mdf:

The scenario I guess I had in mind was some kind of malware that provided remote access to the server (don't know whether this is even possible). The server presumably has all the credentials for all your cloud services it is syncing. With this access, the hacker then either deletes (malicious/psychopath) or changes all the passwords on (ransom) the cloud services and the server itself.

Given I'm still working through options, I'm aiming for something that is as invulnerable as possible (if possible). Right now, to me that means something that is physically isolated from the rest of the internet. Which might only be possible with an external HDD and a bit of personal discipline, but I was wondering if there was a smarter option than that.

AWS Glacier with Vault Lock. It's write once, read many, no delete, and you can prevent any future changes to this policy. It's about as close as you can get to "read only" on the internet.

Of course, if you don't pay your bill, I assume the lot gets deleted eventually.

MikeAqua

8031 posts

Uber Geek
+1 received by user: 3820

#1784300 18-May-2017 15:24

Thanks to the OP for starting this interesting thread.

I'd appreciate some advice for our household.

We have probably 600GB of total data to back up - family photos, videos and important docs.

We both have 32GB surface pros and both travel a lot.

I use OneDrive for anything I'm actively working with.

We archive onto portable hard-drives. One at home and one at work and just keep adding files to them. We need to do better, but it has to be simple and easy and work with low storage computers and travel.

I was thinking about a NAS - back up from personal devices to the NAS and from NAS to the cloud via crash-plan or similar?

Edit: Looks like all the synology consumer grade NAS units support sync to OneDrive, Google Drive, Amazon S3 and bunch of others

?So I could simply get a 1TB plan from one of those providers?

Any advice appreciated.

Mike

timmmay

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784334 18-May-2017 16:14

One hard drive in two locations, offline unless being used, files manually copied, is pretty good. A NAS is always connected which makes it more vulnerable.

Anything can be sent to S3, and from there to Glacier to lower cost. If you use S3 you can only allow writes not deletes, so that could help with preventing viruses deleting, but not sure if you can prevent overwrite. You can use versioning, which only takes up space if you use them. You can also do compressed, de-duplicated backups.

I'm making the distinction between backups and archives. Archives will mostly be images and videos. Archives I'll put into Glacier, backups of current year or working set of documents I'll store on S3 in a deduplicated backup.

MikeAqua

8031 posts

Uber Geek
+1 received by user: 3820

#1784341 18-May-2017 16:23

timmmay:

One hard drive in two locations, offline unless being used, files manually copied, is pretty good. A NAS is always connected which makes it more vulnerable.

Anything can be sent to S3, and from there to Glacier to lower cost. If you use S3 you can only allow writes not deletes, so that could help with preventing viruses deleting, but not sure if you can prevent overwrite. You can use versioning, which only takes up space if you use them. You can also do compressed, de-duplicated backups.

I'm making the distinction between backups and archives. Archives will mostly be images and videos. Archives I'll put into Glacier, backups of current year or working set of documents I'll store on S3 in a deduplicated backup.

I mainly trying to guard against theft of devices, hard-drives etc.

And to have something that just runs in the background without having to remember to bring home and update drives.

Also I have had a portable hard-drive die on me before and lost some photos that way.

Mike

alasta

6891 posts

Uber Geek
+1 received by user: 3365

Trusted

Subscriber

#1784345 18-May-2017 16:42

For those of you who are backing up incrementally to the cloud, do you ever worry about how much data you would have to download if you ever need to do a full restore?

I would be quite happy to upload small amounts daily to keep my backups current, but downloading hundreds of gigabytes in a restore scenario would not be viable on my internet connection.

timmmay

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784346 18-May-2017 16:43

MikeAqua:

I mainly trying to guard against theft of devices, hard-drives etc.

And to have something that just runs in the background without having to remember to bring home and update drives.

Also I have had a portable hard-drive die on me before and lost some photos that way.

CrashPlan for immediate backup. AWS Glacier each year for long term archiving. That's what I'm doing.

Lenovo

Shop now for Lenovo laptops and other devices (affiliate link).

timmmay

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784357 18-May-2017 17:11

alasta:

For those of you who are backing up incrementally to the cloud, do you ever worry about how much data you would have to download if you ever need to do a full restore?

I would be quite happy to upload small amounts daily to keep my backups current, but downloading hundreds of gigabytes in a restore scenario would not be viable on my internet connection.

If you really need to download the backup you'll find a way. Get a fast connection and it'll be down in a few hours.

richms

29104 posts

Uber Geek
+1 received by user: 10217

Trusted

Lifetime subscriber

#1784418 18-May-2017 19:09

I will never need the whole lot in one go if I lost it all. Grab as I need and leave it runny the rest of the time till it's done.

Richard rich.ms

ripdog

548 posts

Ultimate Geek
+1 received by user: 373
Inactive user

#1784496 18-May-2017 22:18

I switched from Crashplan to Arq on Google Drive after crashplan was unable to backup a 700GB file over the course of a week (gigabit fibre). Crashplan is fine for small files, even in large quantities, but it just choked on that one file. So, I had to go through the lengthy process of identifying all the deleted files which were still on Crashplan which I had to restore and allow Arq to push to google drive.

As I was doing restores from crashplan, I discovered a major issue - on large-ish file restores (~20gb or so) it would mysteriously fail in the middle of the file and have to restart. Again. Again. Again. Continuously for several weeks, possibly up to a month. At some point it did succeed, but jeez man!

Arq+gdrive is really nice. It's faster than crashplan, chewed that 700gb file at a steady pace and finished it in about 2.5 days, and has no mandatory rules for retaining backups. Crashplan will delete your backups if you don't use them for a year, and if you remove a local folder from backups, it is deleted from the server as well. Why can't I keep my backups for files I've deleted, but stop them cluttering up my backup list?

The arq developer is really good too, he fixed a bug I had within 2 weeks, and that was when he said he was swamped with work. Good luck getting crashplan to tell you anything more than "read our support articles".

ripdog

548 posts

Ultimate Geek
+1 received by user: 373
Inactive user

#1784523 19-May-2017 01:40

For some reason I'm not allowed to edit a comment I made 3 hours ago, so time for a double post.

There are some very unpleasant rumors floating around the net about crashplan's immediate future. They basically say that the home edition of their product will be shut down on June 1st. Just a rumor of course, so take it with a grain of salt, but worth reading into and being ready for.

https://www.reddit.com/r/Crashplan/comments/6avjb9/code42_ending_crashplan/

https://www.reddit.com/r/DataHoarder/comments/6brlnk/crashplan_ending_personal_accounts/

Within the next few months Code42 will be announcing the end of their CrashPlan for Home product. They have been searching for a buyer to arrange some kind of discount for the customers they are dumping, but I don't think they've found one. They will be proceeding regardless. Customers who don't will keep backing up until the license ends, but will not get software updates or support, and they won't renew your subscription. They are going to take a hard stance about refunds once they announce it public so if you have alot of files backed up, start downloading them now and get a refund before they do. I was a contractor there in support and when I left they were aiming for June 1st.

timmmay

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784526 19-May-2017 05:41

If that happens it'll annoy some people, including me, who's set people up on it and would have to migrate them. But it's just a rumor.

Dyson

Shop now for Dyson appliances (affiliate link).

amanzi

Amanzi
1355 posts

Uber Geek
+1 received by user: 332

ID Verified

Trusted

Lifetime subscriber

#1784574 19-May-2017 08:28

ripdog:

I switched from Crashplan to Arq...

Arq looks really good - do you use it on Mac or Windows? The features seem to tick all the boxes I want from a backup product: https://www.arqbackup.com/features/

MikeAqua

8031 posts

Uber Geek
+1 received by user: 3820

#1784600 19-May-2017 09:41

timmmay:

CrashPlan for immediate backup. AWS Glacier each year for long term archiving. That's what I'm doing.

Will Crashplan work across a couple of computers?

Does it run in the background one set up or does it require pro-active effort by users?

Mike

timmmay

20859 posts

Uber Geek
+1 received by user: 5350

Trusted

Lifetime subscriber

#1784602 19-May-2017 09:44

MikeAqua:

Will Crashplan work across a couple of computers?

Does it run in the background one set up or does it require pro-active effort by users?

I think you can have multiple computers on an account. Alternately you can back up from one computer to another using CrashPlan, but that creates an encrypted archive. I use BitTorrent Sync to mirror files over, then CrashPlan to back them up.

CrashPlan runs in the background and doesn't require any user actions.

MikeAqua

8031 posts

Uber Geek
+1 received by user: 3820

#1784612 19-May-2017 10:05

timmmay:

I think you can have multiple computers on an account. Alternately you can back up from one computer to another using CrashPlan, but that creates an encrypted archive. I use BitTorrent Sync to mirror files over, then CrashPlan to back them up.

CrashPlan runs in the background and doesn't require any user actions.

Thanks we both travel a lot so not often on the same network with both computers turned on. Definitely need to back up machines independently.

Mike

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9