Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your SATA RAID has a 56% chance of data loss

alumnit.ca

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Your SATA RAID has a 56% chance of data loss

#11

Post by lallysingh »

spydez wrote: post #4
patrickg-zill wrote: post #2

ZFS however would protect against this, in that it checksums each chunk of data. And "zfs scrub poolname" will go through and do the checksums for you in the event you are storing a lot of data and not necessarily reading/writing it often.

Well, maybe I should look into transition my home server from Ubuntu and XFS to Nexenta or OpenSolaris and ZFS... I'd hate to lose some of the data on that thing, though I'm currently not doing any RAID. Just a disk of important stuff rsnapshotted every 4 hours to another disk.

I've been running opensol/zfs for almost a year now. Zero problems, good perf & reliability. My research is on two mirrored sata/160gb drives. The most important stuff is on a FS on there with copies=2, making 2 copies on each drive.

Also, backing up zfs is really, really easy. 48 lines of shell scripts do daily incrementals and weekly snapshot backups.

But, do yourself a favor and put /usr/gnu/bin first in your path. oh, and alias tar=gtar. Old solarisisms will otherwise be irritating.

Re: Your SATA RAID has a 56% chance of data loss

#12

Post by JoelSutherland »

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure.

Are any manufacturers offering a package that contains multiple hard drives but behaves as one?

If performance isn't a factor, is there something I don't understand that makes this impossible?

Re: Your SATA RAID has a 56% chance of data loss

#13

Post by wmf »

JoelSutherland wrote: post #12

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure. Are any manufacturers offering a package that contains multiple hard drives but behaves as one? If performance isn't a factor, is there something I don't understand that makes this impossible?

Drobo? Time Capsule?

Re: Your SATA RAID has a 56% chance of data loss

#14

Post by wmf »

attack wrote: post #10
lonestar wrote: post #8

Earlier quoted context omitted.

Why? Assuming you're using CBC mode within each sector, you wouldn't lose any additional data due to a sector error compared to an unencrypted disk.

These are bit level errors. So instead of losing a bit or two, you lose a massive chunk. That's the difference between a strange character in your Doc file and losing the whole thing. I think it's 128 bits for truecrypt, although internet searches point to many people using higher (1024 and 4096). And if the bit level error rates are as high as he says, looks bad.

If the hard disk detects the bit error, it will return an I/O error instead of the corrupt data. Likewise, ZFS will not return corrupt data. So there are plenty of cases where bit errors are promoted to block errors. I can understand the desire to minimize propagation of corruption, but I'd rather just use RAID.

Re: Your SATA RAID has a 56% chance of data loss

#15

Post by briansmith »

JoelSutherland wrote: post #12

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure. Are any manufacturers offering a package that contains multiple hard drives but behaves as one? If performance isn't a factor, is there something I don't understand that makes this impossible?

* Most hard disks are actually multiple hard disk platters packaged together. Even with today's systems, the filesystem can multiplex data across the platters so that the platters mirror each other. Even on a single-platter drive, the file system could just mirror the data on the same platter.

* People want increasingly-higher-capacity, smaller, cheaper, thinner, lighter, and cooler laptops. Adding a second magnetic hard drive into a laptop runs counter to all those goals, as does the kind of mirroring described above. The consumer market demands higher capacity over safety.

* Solid state hard drives seem to be the future, at least for laptops. It might be practical to install dual SSDs into a laptop since they are smaller and lighter. It might even be practical to have a rotating set of three as a backup system. Eject hard disk #1, then insert hard disk #3. Hard disk #1 becomes the backup. Hard disk #3 gets synced with hard disk #2 until they mirror each other. Then, relabel the disks 1->2, 2->3, 3->1.

* People are being encouraged to save (or at least back up) their data on the network if it is really important.

Re: Your SATA RAID has a 56% chance of data loss

#16

Post by JoelSutherland »

wmf wrote: post #13
JoelSutherland wrote: post #12

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure. Are any manufacturers offering a package that contains multiple hard drives but behaves as one? If performance isn't a factor, is there something I don't understand that makes this impossible?

Drobo? Time Capsule?

Drobo is along the lines of what I am looking for. (I use a D-Link DNS 323)

Instead though, I want it to have a SATA interface so the problem would be solved in hardware. Time Capsule fails if the disk fails (and backups aren't stored elsewhere).

Re: Your SATA RAID has a 56% chance of data loss

#18

Post by JoelSutherland »

briansmith wrote: post #15
JoelSutherland wrote: post #12

I keep waiting for a product that solves the failing hard drive problem at the consumer level. For most people I know, the most likely cause of data loss is hardware failure. Are any manufacturers offering a package that contains multiple hard drives but behaves as one? If performance isn't a factor, is there something I don't understand that makes this impossible?

* Most hard disks are actually multiple hard disk platters packaged together. Even with today's systems, the filesystem can multiplex data across the platters so that the platters mirror each other. Even on a single-platter drive, the file system could just mirror the data on the same platter. * People want increasingly-higher-capacity, smaller, cheaper, thinner, lighter, and cooler laptops. Adding a second magnetic…

"The consumer market demands higher capacity over safety."

My contention is that this is beginning to change. Short of video (and at the risk of being wrong in 5 years) there is nothing to do with a 500GB drive.

I think given the trade-off, consumers would choose safety.

Re: Your SATA RAID has a 56% chance of data loss

#20

Post by iigs »

patrickg-zill wrote: post #2

ZFS however would protect against this, in that it checksums each chunk of data. And "zfs scrub poolname" will go through and do the checksums for you in the event you are storing a lot of data and not necessarily reading/writing it often.

As theoretically awesome as ZFS is, the math behind the human factors are more important than the math behind the computer failure rates, and this is something I don't believe we can fix with our current development methodologies.

It looks like Joyent was able to rebuild most (all?) of their data in their ZFS data loss situation, but it could have played out drastically differently.

http://discuss.joyent.com/viewtopic.php?id=19430

Post reply on HN