It still wouldn't upload my 1TB in back-ups in an entire month. Amazon Drive back-up completed in 3 days. Their pricing is amazing, but saving money on a back-up solution that doesn't seem as good as the other cloud storage providers is a dangerous game.
Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
81–90 of 190 posts
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#82Earlier quoted context omitted.
Disclaimer: I'm the author of the blog post. :-) > some of the assumptions that Brian was tossing out are not good ones to make. We COMPLETELY welcome other analysis and listing other assumptions. Internally, we argued endlessly about why this or that wasn't totally accurate, and finally decided to publish the math WITH all of our assumptions exposed so you could be the judge. If Amazon wants to publish their assumpt…
Hey Brian! For the record I love that you are transparent about your assumptions, it is really helpful. I have been in very very similar debates, both at NetApp and at Google of all places. Peter Corbett, the guy who invented the dual parity scheme that NetApp uses, wrote a similar analysis as well for Fast '04 [1]. As someone who likes to geek out on failure proof systems and perfectly secure systems, neither of whi…
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#83Hum, I would be more reassured by past statistics than a probability evaluation. Did they happen to have loss data since their creation?
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#84This analysis is simplistic. Correlated failures are common in drives. That could be a power surge taking out a whole rack, a firmware bug in the drives making them stop working in the year 2038, an errant software engineer reformatting the wrong thing, etc. When calculating your chance of failure, you have to include that, or your result is bogus. Eg. Model A of drive has a failure rate of 1% per year, but when fail…
This is why, when I was building DIY arrays for startups (around the same time Backblaze published their first pod design [1]), I went through the extra effort of sourcing disks from as many different vendors as possible.
Although it was somewhat more time consuming and limited how good a price I could get and how fast the delivery could be, it meant that, for any given disk drive size, it meant I could build an array as large as 12 where no 2 drives were identical in model and manufacturing batch [2].
Of course, it's still a vanishingly rare risk, and "nobody" cares about hardware any more. It does help to remember, at least once in a while, that, on some level, cloud computing really is "someone else's servers" and to hope that someone else still maintains this expertise.
[1] though I used SuperMicro SAS expander backplane chasses for performance reasons
[2] and firmware from the factory, although this is somewhat irrelevant, as one can explicitly load specific firmware versions, and, IIRC, the advantages of consistent firmware across drives, behind a hardware RAID card, outweighed the disadvantages
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#85is the website down?
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#86Earlier quoted context omitted.
> Can this be reformulated Roughly, yes. > Also curious what are the stats from other providers. As to other providers, most are 6+ 9s that I’ve looked at, with many in the 8-9 range. Anything over 8 is (as they admitted) essentially marketing porn and not a useful metric (for reasons they mentioned as well as ones said by other comments here).
And yet, no much complains about lost GMail letters in internet because of disc failures.. And GMail is likely stores much more than 10 trln objects and 100TB of data.
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#87Earlier quoted context omitted.
> Can this be reformulated Roughly, yes. > Also curious what are the stats from other providers. As to other providers, most are 6+ 9s that I’ve looked at, with many in the 8-9 range. Anything over 8 is (as they admitted) essentially marketing porn and not a useful metric (for reasons they mentioned as well as ones said by other comments here).
And yet, no much complains about lost GMail letters in internet because of disc failures.. And GMail is likely stores much more than 10 trln objects and 100TB of data.
I think a much bigger scandal is that all major laptop Operating System vendors (Microsoft and Apple) absolutely know when your laptop drive loses files or even is starting to go bad in some cases, and they NEVER tell the customer. I think an excellent product offering would be a 3rd party piece of software and cloud service which was a "verification service". It wouldn't store your files offsite, it would store the name, size, and SHA1 offsite and periodically check that no bits have been flipped on your local drive unless you intended it. For example, a week after I take a photo, I absolutely never want the photo to change. Ever. Same with music I (legally) download.
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#88This analysis is simplistic. Correlated failures are common in drives. That could be a power surge taking out a whole rack, a firmware bug in the drives making them stop working in the year 2038, an errant software engineer reformatting the wrong thing, etc. When calculating your chance of failure, you have to include that, or your result is bogus. Eg. Model A of drive has a failure rate of 1% per year, but when fail…
Which means every customer suffers data loss They suffer partial backup loss. The customer only suffers data loss if they lost their 'master' copy of the data as well during the outage. Iff they don't have a secondary backup solution.
If it's the latter, "data loss" seems an appropriate characterization.
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#89It still wouldn't upload my 1TB in back-ups in an entire month. Amazon Drive back-up completed in 3 days. Their pricing is amazing, but saving money on a back-up solution that doesn't seem as good as the other cloud storage providers is a dangerous game.
Where are you geographically? Was your Amazon upload to a datacenter physically much nearer to you than California? 30MBit/s sustained for 3 days isn't unreasonable for a business connection, but seems high compared to most of what I see available at least in the US.
> 30MBit/s sustained for 3 days isn't unreasonable for a business
We (Backblaze) are seeing more and more consumer internet connections in the USA with 20 Mbit/sec upstreams, I thought they were available most everywhere if you were willing to upgrade your internet package "just a bit". 30 Mbits is a little unusual for "consumer", but not unheard of. Of course, there is a "selection bias" when you look at online backup users. :-)
Re: Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter
#90Oh gosh thanks Backblaze, I'll just dig through several TB of stuff....