Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

backblaze.com

Post reply on HN
Print view View original

91–100 of 190 posts

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#91

Post by schoen »

matt4077 wrote: post #38
schoen wrote: post #3

Earlier quoted context omitted.

I wonder if there's a general term in engineering for the case where a particular risk has been reduced well below the likelihood of more serious but exotic risks. I've heard about this most in cryptography where we can sometimes say that the risk of, say, a system failing due to an unexpected hash collision is drastically less than the risk of the system failing due to cosmic radiation or various natural disasters.…

It’s not a perfect match, but Rumsfeldian “unknown unknowns” come to mind. Specifically: every X-nines durability design will be compromised by some failure mode you didn’t think of.

I don't think unknown unknowns are what I'm thinking of here. In this case the argument involves a very specific risk, and sometimes a very specific lower bound for its probability.

For example, in the hash collision case the argument says that it's not worth worrying about the (known) probability of a software error due to an unexpected hash collision because it's dominated by the (known) probability of a comparable error due to cosmic radiation. (The former probability can be calculated using the birthday paradox formula, and the latter has been characterized experimentally in different kinds of semiconductor chips.)

This kind of argument doesn't rely on the idea that there are other risks that we can't identify or quantify. It's about comparing two failure modes that we did think of, in order to argue that one of them is acceptable or at least not worth further attempts to mitigate.

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#95

Post by k__ »

brianwski wrote: post #58
eloff wrote: post #5

This was an interesting read from a technical point of view, but also well written and refreshingly transparent. I found the discussion about why it doesn't matter when you start talking about 11 nines of reliability to be hilariously true. At the end of the day we're still flawed humans living in a hostile universe, and no matter how foolproof we make the technology, there are some weaknesses that just can't be elim…

Disclaimer: I'm the author of the blog post. :-) > well written and refreshingly transparent Thank you! In the interests of full transparency, the blog post was a collaborative affair and was proof read and edited for clarity by several people at Backblaze. > discussion about why it doesn't matter One of the philosophies Backblaze uses is to build a reliable component out of several inexpensive and unrelated componen…

BB and S3 both have eleven 9 durability, how much does using both increase this?

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#96

Post by minitoar »

tjoff wrote: post #62
klodolph wrote: post #7

Earlier quoted context omitted.

I've posted the math here before but if we assume that an asteroid hits the earth every 65 million years and wipes out the dominant life forms, then this fact alone puts your yearly durability at a maximum of ~8 nines. The point about billing is better, though. My other concern is that a software bug, operator error, or malicious operator deletes your data.

That's why no sane entity would use one earth. Use two and you can quickly recover.

If you have two you have one, if you have one you have none.

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#97

Post by fencepost »

brianwski wrote: post #89
fencepost wrote: post #81

Earlier quoted context omitted.

Where are you geographically? Was your Amazon upload to a datacenter physically much nearer to you than California? 30MBit/s sustained for 3 days isn't unreasonable for a business connection, but seems high compared to most of what I see available at least in the US.

Disclaimer: I work at Backblaze and live in California. > 30MBit/s sustained for 3 days isn't unreasonable for a business We (Backblaze) are seeing more and more consumer internet connections in the USA with 20 Mbit/sec upstreams, I thought they were available most everywhere if you were willing to upgrade your internet package "just a bit". 30 Mbits is a little unusual for "consumer", but not unheard of. Of course,…

Yes regarding the available speed, but on a consumer connection with Comcast at least if you push 30MBit for 3 days straight you may get a call or at least may start seeing popups in your browser on any http (not https) traffic.

From Comcast: The Terabyte Internet Data Usage Plan is a new data usage plan for XFINITY Internet service that provides you with a terabyte (1 TB or 1024 GB) of Internet data usage each month as part of your monthly service. If you choose to use more than 1 TB in a month, we will automatically add blocks of 50 GB to your account for an additional fee of $10 each. Your charges, however, will not exceed $200 each month, no matter how much you use. And, we're offering you two courtesy months, so you will not be billed the first two times you exceed a terabyte.

Also, All customers in locations with an Internet Data Usage Plan receive a terabyte per month, regardless of their Internet tier of service. and The data usage plan does not currently apply to XFINITY Internet customers on our Gigabit Pro tier of service. The plan also does not apply to Business Internet customers, customers on Bulk Internet agreements, and customers with Prepaid Internet.

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#98

Post by bhelkey »

k__ wrote: post #95
brianwski wrote: post #58

Earlier quoted context omitted.

Disclaimer: I'm the author of the blog post. :-) > well written and refreshingly transparent Thank you! In the interests of full transparency, the blog post was a collaborative affair and was proof read and edited for clarity by several people at Backblaze. > discussion about why it doesn't matter One of the philosophies Backblaze uses is to build a reliable component out of several inexpensive and unrelated componen…

BB and S3 both have eleven 9 durability, how much does using both increase this?

Depends what you are modeling. The probabilities of random disk failures, are probably independent.

However, there are risks that are not necessarily independent such as the US Government ordering these two services to delete your data, or, as the article mentions, an armed conflict destroying data centers.

Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter

#100

Post by hinkley »

londons_explore wrote: post #9

This analysis is simplistic. Correlated failures are common in drives. That could be a power surge taking out a whole rack, a firmware bug in the drives making them stop working in the year 2038, an errant software engineer reformatting the wrong thing, etc. When calculating your chance of failure, you have to include that, or your result is bogus. Eg. Model A of drive has a failure rate of 1% per year, but when fail…

You could also have a bad batch of drives, causing a bunch of failures to happen over the period of a couple of months. Sources of failure don't take a number. Like in your example, they can and will overlap, which is why when we try to design bullet proof systems we aren't satisfied until every bad even requires a number of separate things to go wrong all at once.

But if you have faulty power or bad hardware chipping away at your equipment, your depth of resiliency is degraded until the issue is corrected.

Post reply on HN