Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze hard drive reliability stats for Q3 2016

backblaze.com

Post reply on HN
Print view View original

81–90 of 116 posts

Re: Backblaze hard drive reliability stats for Q3 2016

#81

Post by Retric »

pfarnsworth wrote: post #17

With hard drive size increasing so quickly but hard drive transfer speeds basically flat, I wonder if there are long-term implications for them with respect to recovery from backup and downtime. For example, if a whole rack goes down, and they are on 32TB drives in the future, for example, could it takes a week or more for their data to get online?

Transfer increases as square root of drive density. http://www.anandtech.com/show/10541/western-digital-adds-hel... Shows 10GB drives at 249 MB/s sustained sequential transfer rate.

That takes ~11 hours to fill. At 400x the density it would take 20x as long or 9 days. I don't think HDD drives are hitting 400x the density any time soon but if they did it would be a problem.

However, in an array you could take a month to fill a drive to 75% without causing to much trouble. Assuming you had enough drives. That's around a ~80PB limit drive. IMO, the real issue is it would take another month to download all that data. Relegating HDD firmly into archival storage.

PS: I don't think rust is going to get into those density's making this far less of an issue.

Re: Backblaze hard drive reliability stats for Q3 2016

#82

Post by mapt »

dom0 wrote: post #40
pfarnsworth wrote: post #17

With hard drive size increasing so quickly but hard drive transfer speeds basically flat, I wonder if there are long-term implications for them with respect to recovery from backup and downtime. For example, if a whole rack goes down, and they are on 32TB drives in the future, for example, could it takes a week or more for their data to get online?

Well, uh, yes. No one really expects that rotational rust will get much faster, and in fact history shows that, compared to the increase in density, the increase in transfer rates are laughable at best. Between 1990 and today you are probably looking at a 20 000 times increase in density, yet transfer rates only increased around a factor of around 150-200. [In fact, from the early 1960 to today it's only a factor of…

Or more read-write heads?

Re: Backblaze hard drive reliability stats for Q3 2016

#84

Post by eye2sky »

mahyarm wrote: post #75
dom0 wrote: post #40

Earlier quoted context omitted.

Well, uh, yes. No one really expects that rotational rust will get much faster, and in fact history shows that, compared to the increase in density, the increase in transfer rates are laughable at best. Between 1990 and today you are probably looking at a 20 000 times increase in density, yet transfer rates only increased around a factor of around 150-200. [In fact, from the early 1960 to today it's only a factor of…

How about the stupid idea of making a minimum block size be a multiple of the number of platters to write to. The block is divided evenly amongst all platters and are always at the same parallel track/sector. That way you can multiply the read/write speeds to be a function of the number of platters. Instead of 50-100MB/s, you can get 4-8x the speed in large linear transfers, which helps get dead racks back up faster…

You cannot read from multiple platters in parallel. When you are aligned to read from a certain track on a certain side of a platter, you are not necessarily aligned on all platters. This head mechanism is not accurate enough for that.

Re: Backblaze hard drive reliability stats for Q3 2016

#85

Post by NicoJuicy »

atYevP wrote: post #32
MrMullen wrote: post #30

Earlier quoted context omitted.

You should really consider moving this webpage and report to something like AWS S3 when you first release it. Then move back to your usual servers when traffic has fallen off. Your poor servers must melt down when this shows up on Hacker News and Slashdot.

Internally we're blaming our SEO people for putting to much crap on the blog itself ;) But yea, it's worth exploring - though we have our own servers that should be able to handle the load. We haven't had blog loading trouble in a while, so it'll be neat to debug this later :D

EasyEngine for nginx with redit cache and php7 Will set your wordpress blog blazing.

Yoast is a culprit of performance though.

Don't forget the plugin query monitor and http2 doesn't need bundling resources ( I suppose)

Re: Backblaze hard drive reliability stats for Q3 2016

#86

Post by e40 »

leejoramo wrote: post #8

How are people using Backblaze's excellent hard drive reliability reports in making purchasing decisions? For example when I search for HGST HMS5C4040ALE640 on Amazon I get a dealer selling old out of warrantee drives as new. https://www.amazon.com/HGST-MegaScale-HMS5C4040ALE640-Coolsp... I get similar results with many of the other drives listed and with other websites such as NewEgg.

For example when I search for HGST HMS5C4040ALE640 on Amazon I get a dealer selling old out of warrantee drives as new.

We found this, as well. And we stopped trying to get the HGST drives after we got a bad batch from a seller on Amazon.

Re: Backblaze hard drive reliability stats for Q3 2016

#88

Post by kikoreis »

brianwski wrote: post #54
fpp wrote: post #43

Earlier quoted context omitted.

I'll see from your reports that you're migrating 1000s of 2TB drives to 8TB drives - what is actually happening to your old 2TB drives? Guess you are not throwing those away so what are you doing with them.

Disclaimer: I work at Backblaze. We securely wipe the drives, then we sell them to a "used hard drive reseller".

Does "secure wipe" mean ATA SE, i.e. hdparm --security-erase, or --security-erase-enhanced?

Re: Backblaze hard drive reliability stats for Q3 2016

#89

Post by kikoreis »

brianwski wrote: post #56
Llevel wrote: post #28

Earlier quoted context omitted.

I chose to get 4 ST4000DM000 drives based on previous reports. Sure HGST drives never die, but it's cheaper to RMA or buy a single new drive if one fails, than the added cost of 4 reliable drives. Assuming only one fails, which is a risk I'm willing to take with my very non-mission-critical data. I don't read this as 'which drive to buy' but more as 'which drive not to buy'.

> HGST drives never die Disclaimer: I work at Backblaze. I know you didn't mean that as an absolute, but I just want to point out 100% of drives fail. It's my OCD that makes me point this out. We have NEVER found a drive that lasted forever. There are two types of drives: 1) those that have already failed, and 2) those that are about to fail. For any data you would be annoyed to lose, you need three copies in three l…

To be somewhat similarly OCD, if your software is controlling storage to each of those drives, then there are many shared LOC involved; indeed, this is the reason why async replication into separate clusters is always recommended for data which can't be lost.

Re: Backblaze hard drive reliability stats for Q3 2016

#90

Post by atYevP »

meritt wrote: post #83

We have about 20TB in an AWS S3 bucket we'd like to backup somewhere separate from Amazon. Is there any chance of Backblaze offering ingestion from an Amazon Snowball export ( https://aws.amazon.com/snowball/ )?

Yev from Backblaze here -> Believe Snowball is encrypted and fairly locked down, not sure we could do that.
Post reply on HN