Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HGST gets closer to shipping 10TB HDD

zdnet.com

Post reply on HN
Print view View original

41–50 of 66 posts

Re: HGST gets closer to shipping 10TB HDD

#41

Post by Viper007Bond »

fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

That seems slow. I have a 6 x 6TB RAID 5 array (27TB) for non-critical data storage and it takes less something like 22 hours to do its weekly parity check. I'm running firmware RAID via an onboard Intel controller on an ASUS board.

Re: HGST gets closer to shipping 10TB HDD

#42

Post by rasz_pl »

ghshephard wrote: post #18
eck wrote: post #15

Earlier quoted context omitted.

> the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. Well there's one issue that another whole drive will fail and you're screwed. The other issue is that modern disks have an unrecoverable read error rate compared to their size such that a total cover-to-cover read -- necessary on every remaining disk to rebuild a RAID5 -- is kinda unreliable, even with a supposedly healthy dis…

I'm interested - do you have a citation for that? I'm wondering if manufacturers of large drives accommodate for the statistically increased chance of a cover-cover failure (based on having so much data) by increasing their redundancy data/checksums to keep it constant.

fun fact: Commodore Amiga floppies used 32bit XOR checksum = 3% of undetected two bit errors!!

http://www.techtravels.org/amiga/amigablog/?p=280

This is probably because Amiga didnt have real hardware floppy-disk controller, just a general IO (CIA) chips, and read raw serial datastream into ram, all the decoding was done in software. Similar to Apple II 2x 8bit XOR checksum.

Re: HGST gets closer to shipping 10TB HDD

#43

Post by rasz_pl »

ars wrote: post #33
pkaye wrote: post #24

Earlier quoted context omitted.

I see a new feature for SATA called Rebuild Assist becoming available to help reduce the RAID recovery times.

http://www.hgst.com/tech/techlib.nsf/techdocs/CCA92CECECFCC7... The way it works is by realizing there are only a few bad sectors on the failed drive, so rebuild that area first, the used the "bad" drive to help rebuild the rest of the array at full speed.

Sounds clever. Too bad SSD (due to being shit and simply dying all at once) will never support that.

Re: HGST gets closer to shipping 10TB HDD

#44

Post by static_noise »

eck wrote: post #15
fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

> the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. Well there's one issue that another whole drive will fail and you're screwed. The other issue is that modern disks have an unrecoverable read error rate compared to their size such that a total cover-to-cover read -- necessary on every remaining disk to rebuild a RAID5 -- is kinda unreliable, even with a supposedly healthy dis…

> Well there's one issue that another whole drive will fail and you're screwed.

You do know that in a RAID6 two drives may fail without causing data loss? In any way you need one or more hotspares ready plugged in in order to keep the time window short.

RAID6 is not perfect but the probability of 3 drives failing during the rebuilt window is much lower than the probability of 2 drives failing. (One also has to consider errors such as memory corruption, chip failures or catastrophic failure to the power supply where no RAID level will protect you from.)

A RAID1 built of two RAID6s may be necessary to avoid performance drops during rebuild. In the case of multiple failures a RAID6+1 setup will protect you from at least 4 hard drives failing in the rebuild time window.

> The other issue is that modern disks have an unrecoverable read error rate compared to their size such that a total cover-to-cover read -- necessary on every remaining disk to rebuild a RAID5 -- is kinda unreliable, even with a supposedly healthy disk.

This is another reason for a RAID6. Not only does it recover when one or two disks fail. It also recognizes and recovers broken blocks when one disk returns the wrong data. You scrub the disk weekly and remap broken sectors or swap out (soon to break) harddrives.

Re: HGST gets closer to shipping 10TB HDD

#46

Post by jacquesm »

moe wrote: post #8
fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. Two disk failures in the same array within 15 hours is very rare.

Frequent enough that I've seen it twice on different arrays. The reason for this is that a drive failure can be caused by an external factor (usually multiple drives fed by the same broken power supply, but not yet so broken that it goes off-line).

Re: HGST gets closer to shipping 10TB HDD

#48

Post by gtwy »

jacquesm wrote: post #34
letstryagain wrote: post #28

Earlier quoted context omitted.

We stopped using RAID5/6 several years ago because of rebuild time and associated performance drop. Now with these massive disks we don't even use RAID10 any more! Every disk is just paired with one mirror and that's it. (RAID1) The plan is to move to JBOD soon and just use software to store each file on 3 random disks across 3 separate servers.

That creates a substantial window during which you could lose the second drive.

I'm sure they have their reasons (I won't pretend to know every detail about their setup) but it's not a solution I would adopt in our environment.

Re: HGST gets closer to shipping 10TB HDD

#49

Post by sitkack »

ghshephard wrote: post #13
fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

This is probably why people like BackBlaze use RAIN (Redundant Array of Inexpensive Nodes) instead of RAID. Also, their drive failure is 10 disks/day out of a population of 44,100 drives, about 0.02%. So, between the lack of correlation of failure (their Nodes, or, "Pods" are in different racks), their ability to lose an entire POD regardless (they have 3 parity pods), and the relatively low disk failure rate - the l…

There is a very interesting engineering optimization strategy at play here, would love to do/see a blog post on it.

Simple structures, with a large safety margin and simple control flow vs complex structures, very light weight but at the limits of capacity.

Like an ethernet switch doesn't need complex flow routing algorithms when the raw backplane bandwidth is some multiplier higher than will ever experience contention.

It always seems to me, that things built near their design limits will eventually expose a catastrophic flaw.

Human engineered items that have stood the test of time, Roman Aqueducts, Brooklyn Bridge, DC3, Dodge Dart, Toyota Corolla, AK47, PDP8 all have commonalities in their design philosophies.

Re: HGST gets closer to shipping 10TB HDD

#50

Post by TheCondor »

moe wrote: post #8
fiatmoney wrote: post #4

The 8 TB HDDs available now take a full 15 hours or so for a linear scan at max speed. God help you if you ever need to rebuild a RAID array made of them; the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. We're nearing the point at which the throughput relative to the capacity and the risk of failure or corruption makes further capacity less and less useful, unless you're doing…

the risk of an additional failure in say a RAID5 or RAID6 configuration is just too high. Two disk failures in the same array within 15 hours is very rare.

This is true of natural age type failures. When in a RAID every drive is effectively experiencing the same experiences, down to the writes and reads, if the failure is due to the drive's interaction with its environment then one failure almost always predicts more failures.

Natural age failures are pretty rare.

Post reply on HN