Your SATA RAID has a 56% chance of data loss • Hacker News Board

Re: Your SATA RAID has a 56% chance of data loss

#21

Post by briansmith » Tue, Sep 09, 2008, 12:12 AM UTC

JoelSutherland wrote: post #18

briansmith wrote: post #15

Earlier quoted context omitted.

* Most hard disks are actually multiple hard disk platters packaged together. Even with today's systems, the filesystem can multiplex data across the platters so that the platters mirror each other. Even on a single-platter drive, the file system could just mirror the data on the same platter. * People want increasingly-higher-capacity, smaller, cheaper, thinner, lighter, and cooler laptops. Adding a second magnetic…

"The consumer market demands higher capacity over safety." My contention is that this is beginning to change. Short of video (and at the risk of being wrong in 5 years) there is nothing to do with a 500GB drive. I think given the trade-off, consumers would choose safety.

For the same price point, maybe. But, hard drives are one of the most expensive components of a laptop. Choosing between $500 and $600 is a big deal. Network backup like Mozy is competitively priced, especially when you consider the high chance of catastrophic loss that laptops have (stolen, lost, dropped, flooded).

Plus, if you were going to trade 50% of capacity for safety, there are a lot of other kinds of safety precautions to consider. For example, file history versioning (like a local, always-on Mozy or Dropbox or Subversion) would probably protect the user from more data loss than RAID, because most data loss is due to human error, not mechanical failure.

That said, I would really like my laptop to have RAID-1. Actually, I have two 100GB hard drives and an Ultrabay...maybe I will give it a shot on my Linux partition.

Re: Your SATA RAID has a 56% chance of data loss

#22

Post by Andys » Tue, Sep 09, 2008, 1:01 AM UTC

ars wrote: post #7

Wow. If you have an 8TB raid, that's 7TB of data so 61,572,651,155,456 bits A hard disk has a bit error rate of 10^15: 1,000,000,000,000,000 That's about 16 times higher. That's horrible. Taken to a single 750GB disk that means you can read the entire disk an average of 155 times before you'll encounter an error! That's shockingly bad.

I am, among other things, a sysadmin who babysits several RAIDs, including a 10TB one.

I can confirm that the large capacity SATA disks have a noticeably high error rate, which I have never seen before. It used to be that RAID protected you from disk failures, but we aren't seeing that nearly as much now as we are unrecoverable bit errors on otherwise healthy disks.

I have set all our RAID controllers to scrub the data and compare the mirrors every weekend. In the past couple of years we've only had a couple of unrecoverable read errors, but many recoverable read errors.

Recoverable read errors are still bad, the disk gets taken offline and out of the RAID set for up to a minute while the controller retries many times until the data can be read. Then the sector is remapped to another part of the disk.

So yes, it is a real problem, but it isn't too hard to manage in this era of filesystem snapshotting plus high speed RAID6 controllers. In return we've been given huge amounts of storage space.

Re: Your SATA RAID has a 56% chance of data loss

#23

Post by wmf » Tue, Sep 09, 2008, 1:04 AM UTC

JoelSutherland wrote: post #16

wmf wrote: post #13

Earlier quoted context omitted.

Drobo? Time Capsule?

Drobo is along the lines of what I am looking for. (I use a D-Link DNS 323) Instead though, I want it to have a SATA interface so the problem would be solved in hardware. Time Capsule fails if the disk fails (and backups aren't stored elsewhere).

I want it to have a SATA interface so the problem would be solved in hardware.

I don't see much difference between SATA and Firewire; in both cases Drobo is hardware RAID.

Time Capsule fails if the disk fails (and backups aren't stored elsewhere).

But then you just make a new backup.

To protect against disk failure, you can use backup or RAID. (RAIDed backup is a little expensive for personal use.) Backup also protects against file deletion or corruption.

Re: Your SATA RAID has a 56% chance of data loss

#24

Post by thorax » Tue, Sep 09, 2008, 1:11 AM UTC

JoelSutherland wrote: post #12

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure. Are any manufacturers offering a package that contains multiple hard drives but behaves as one? If performance isn't a factor, is there something I don't understand that makes this impossible?

Windows Home Server actually uses a duplication scheme that makes sure all your machine backups (and shared folders) are stored, in full, on more than one drive on the server. It behaves pretty much like one big drive and duplicates across all the drives as needed.

It's pretty likely a consumer won't need more than this since that equates to three copies of your data at any given time (laptop, server disk B, and server disk C).

Re: Your SATA RAID has a 56% chance of data loss

#25

Post by cstejerean » Tue, Sep 09, 2008, 1:29 AM UTC

lonestar wrote: post #8

attack wrote: post #6

This is what keeps encryption impractical, especially for laptops.

Why? Assuming you're using CBC mode within each sector, you wouldn't lose any additional data due to a sector error compared to an unencrypted disk.