Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.




14789 posts

Uber Geek
+1 received by user: 2752

Trusted
Subscriber

# 138285 28-Dec-2013 14:46
Send private message

I've noticed that my larger backup disks that aren't used very often are starting to fail, often just with a single sector being reallocated, then another sector later. I've read this could happen more and more as drives get bigger, single bit or sector errors that cause files to become corrupt. I assume there's some error checking and correction built in at the hardware level, but can a file system be designed to combat this?

You might say treat a 2TB hard drive like a 1TB drive, each byte is written to different areas of the disk. This would give you better reliability, and disks are so cheap the loss of space probably isn't important for most consumers.

Has anything like this been done in a way that's usable for windows machines?

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2 | 3
2346 posts

Uber Geek
+1 received by user: 410

Trusted
Subscriber

  # 958331 28-Dec-2013 15:16
Send private message

That's where SAS disks come into play, overall more reliable, and I believe they have more spare sectors too



14789 posts

Uber Geek
+1 received by user: 2752

Trusted
Subscriber

  # 958337 28-Dec-2013 15:30
Send private message

Spare sectors isn't quite what I mean. I mean keeping two copies of the data on different parts of the same disk. I assume not all errors are recoverable.

 
 
 
 


8272 posts

Uber Geek
+1 received by user: 4560


  # 958347 28-Dec-2013 16:11
Send private message

timmmay: Spare sectors isn't quite what I mean. I mean keeping two copies of the data on different parts of the same disk. I assume not all errors are recoverable.


I get what you mean, but not a good idea IMO.  It might marginally reduce the chance of permanently losing data in case of head crash etc, but because it's going to be writing twice, then that's more wear and tear on the mechanical components, and not giving you any advantage in case of electronic/board failure. I expect if it could be achieved, then it would also be very slow.

1952 posts

Uber Geek
+1 received by user: 513

Lifetime subscriber

  # 958349 28-Dec-2013 16:17
Send private message

I keep multiple copies using file syncing of folders or entire partition. There's a lot of free software to do this e.g. freefilesync, rsync. If data does become corrupted then you don't want the corruption copied to that other partition so you don't want to use a comparison of file contents to start syncing for that file.

There are file systems that provide redundancy and there were some solutions under development for Windows but I'm not sure if they are available yet.

The Wikipedia article on data corruption might point you in the right direction. ZFS, for example, performs continuous integrity checks and can correct most data errors and I use it on my NAS.

[Edit to correct spelling]

488 posts

Ultimate Geek
+1 received by user: 80

Trusted

  # 958381 28-Dec-2013 18:10
Send private message

Hi,

I've seen this on large drives that are infrequently used but have power saving enabled, that is, they are constantly being powered up and down because they are infrequently used.

To avoid problems I've had in the past, I've disabled all forms of power saving and bought enterprise grade drives where possible.

I don't think I've heard of software raid between partitions on the same device, and I'm sure it won't make disks more reliable because you'd be wearing the head down trying to sync the partitions. I know it's not what you want to hear, but you should have two disks, separate power supplies each with a copy of your stuff, minimum.

+1 for ZFS and it's friend RAID-Z - I think Windows 2012 has a similar file/RAID system available when storage services are setup.

Good luck.



14789 posts

Uber Geek
+1 received by user: 2752

Trusted
Subscriber

  # 958432 28-Dec-2013 21:02
Send private message

Thanks for the thoughts guys, doesn't sound very practical. I have my backups on external drives in three separate locations, but I use mirroring a lot so I am vulnerable to corruption being copied. I'm wary of differential backup software, I wouldn't want a single bit image in a 50GB backup file to corrupt everything, but I guess you can't protect against everything.

Maybe I'll just use amazon glacier and let them worry about it. Problem is the 2TB+ of images, gets expensive.

488 posts

Ultimate Geek
+1 received by user: 80

Trusted

  # 958435 28-Dec-2013 21:06
Send private message

What you need is a mate across town who has VDSL or FTTH and leave a spare encrypted disk at his place. It would be free if you offered to return the favour.

 
 
 
 


2457 posts

Uber Geek
+1 received by user: 147


  # 958491 29-Dec-2013 00:19
Send private message

btrfs can do this. (RAID1 on a single disk/block cloning)

https://en.wikipedia.org/wiki/Btrfs#Cloning

844 posts

Ultimate Geek
+1 received by user: 200


  # 958505 29-Dec-2013 05:50
Send private message

Both zfs and btrfs can do what you want on a single drive. (You would have to split it into partitions with zfs, btrfs can do this on a drive with one partition.)

Personally, I would pick zfs - I use it myself.

Neither of those will work on windows.

For a pure windows solution, for backups that need to be protected against single sector disk failures, I would pick the following:

Use winrar to create a rar file of the part of your windows file system that you wish to backup. Rather than have this done as one single large rar file, select the option to split into (say) 100 parts and then write a further (say) 5 recovery volumes.

If you want to get back just one file, open the first rar file with winrar and then navigate the directories until you find the file and then extract it.

If you get sector level corruption on the drive, then as long as you can read 100 files from your set of 100 rar files and five recovery volumes then winrar can restore your full backup. You can deal with as many bad sectors as you have recovery volumes.

In a way this will also get you one of the big advantages that zfs has over something like raid and that is protection against silent corruption*. You will get the same using winrar like this because it checksums all of its data.

I have used exactly this technique on windows systems in the past and have been able to recover from cases where I had lost the maximum allowable number of rar or recovery files. It works very well indeed.

*If you do not know about this, google for descriptions of the raid 5 write hole.



14789 posts

Uber Geek
+1 received by user: 2752

Trusted
Subscriber

  # 958520 29-Dec-2013 09:03
Send private message

Interesting about the file systems, I'll do some reading. I'm not sure how practical those file systems would be given these are for external backup disks that are connected weekly to quarterly.

gundar: What you need is a mate across town who has VDSL or FTTH and leave a spare encrypted disk at his place. It would be free if you offered to return the favour.


This is for backup disks that are connected weekly to quarterly. Online backup isn't practical for the time frames or data volumes in question, I often add 40GB at a backup after photographing an event. That's half a months download allowance.

844 posts

Ultimate Geek
+1 received by user: 200


  # 958527 29-Dec-2013 09:22
Send private message

timmmay: Interesting about the file systems, I'll do some reading. I'm not sure how practical those file systems would be given these are for external backup disks that are connected weekly to quarterly.

gundar: What you need is a mate across town who has VDSL or FTTH and leave a spare encrypted disk at his place. It would be free if you offered to return the favour.


This is for backup disks that are connected weekly to quarterly. Online backup isn't practical for the time frames or data volumes in question, I often add 40GB at a backup after photographing an event. That's half a months download allowance.


You could consider making a usb boot stick with Linux on it and then, come backup time, booting the PC off the stick, mounting an external drive that you have formatted with btrfs and copying the data onto the drive. Linux/btrfs would be a better choice than FreeBSD/ZFS for that as Linux will do better at working with your windows disks. With FreeBSD/ZFS you are better off having that on a stand alone machine and mounting it over the network.

661 posts

Ultimate Geek
+1 received by user: 52


  # 958537 29-Dec-2013 10:14
Send private message

You can run ZFS on Linux ;)

844 posts

Ultimate Geek
+1 received by user: 200


  # 958543 29-Dec-2013 10:37
Send private message

PANiCnz: You can run ZFS on Linux ;)


Is it still run as a user mode file system? Are there drawbacks with that?


BDFL - Memuneh
63598 posts

Uber Geek
+1 received by user: 14079

Administrator
Trusted
Geekzone
Lifetime subscriber

  # 958545 29-Dec-2013 10:43
Send private message

timmmay: Thanks for the thoughts guys, doesn't sound very practical. I have my backups on external drives in three separate locations, but I use mirroring a lot so I am vulnerable to corruption being copied. I'm wary of differential backup software, I wouldn't want a single bit image in a 50GB backup file to corrupt everything, but I guess you can't protect against everything.

Maybe I'll just use amazon glacier and let them worry about it. Problem is the 2TB+ of images, gets expensive.


How much space are you using for backup?








14789 posts

Uber Geek
+1 received by user: 2752

Trusted
Subscriber

  # 958588 29-Dec-2013 12:55
Send private message

Booting to linux isn't very practical. I'd probably go full RAID before I did that. Thanks for the idea though.

freitasm:
timmmay: Thanks for the thoughts guys, doesn't sound very practical. I have my backups on external drives in three separate locations, but I use mirroring a lot so I am vulnerable to corruption being copied. I'm wary of differential backup software, I wouldn't want a single bit image in a 50GB backup file to corrupt everything, but I guess you can't protect against everything.

Maybe I'll just use amazon glacier and let them worry about it. Problem is the 2TB+ of images, gets expensive.


How much space are you using for backup?


Somewhere between 2TB and 4TB, but I don't back up everything to every location.

 1 | 2 | 3
View this topic in a long page with up to 500 replies per page Create new topic



Twitter and LinkedIn »



Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

New Zealand PC Market declines on the back of high channel inventory, IDC reports
Posted 18-Jun-2019 17:35


Air New Zealand uses drones to inspect aircraft
Posted 17-Jun-2019 15:39


TCL Electronics launches its first-ever 8K TV
Posted 17-Jun-2019 15:18


E-scooter share scheme launches in Wellington
Posted 17-Jun-2019 12:34


Anyone can broadcast with Kordia Pop Up TV
Posted 13-Jun-2019 10:51


Volvo and Uber present production vehicle ready for self-driving
Posted 13-Jun-2019 10:47


100,000 customers connected to fibre broadband network through Enable
Posted 13-Jun-2019 10:35


5G uptake even faster than expected
Posted 12-Jun-2019 10:01


Xbox showcases 60 anticipated games
Posted 10-Jun-2019 20:24


Trend Micro Turns Public Hotspots into Secure Networks with WiFi Protection for Mobile Devices
Posted 5-Jun-2019 13:24


Bold UK spinoff for beauty software company Flossie
Posted 2-Jun-2019 14:10


Amazon Introduces Echo Show 5
Posted 1-Jun-2019 15:32


Epson launches new 4K Pro-UHD projector technology
Posted 1-Jun-2019 15:26


Lenovo and Qualcomm unveil first 5G PC called Project Limitless
Posted 28-May-2019 20:23


Intel introduces new 10th Gen Intel Core Processors and Project Athena
Posted 28-May-2019 19:28



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.


Support Geekzone »

Our community of supporters help make Geekzone possible. Click the button below to join them.

Support Geezone on PressPatron



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.