Live data from Hacker News

Backblaze has ordered 100 petabytes of hard drives

backblaze.com

71–80 of 146 posts

Re: Backblaze has ordered 100 petabytes of hard drives

#71
post #39
post #21

Earlier quoted context omitted.

Yev from Backblaze here -> Yes, unless the storage pod is in maintenance, it remains on so that backup and B2 data can be uploaded and accessed.

Interesting. Have you ever computed how much power you would save if you turned them off when not needed? Or is every pod constantly in use? Would it help if smart algorithms repartition the data on storage pods, to optimize power use?

Brian from Backblaze here.

> Have you ever computed how much power you would save if you turned them off when not needed?

Electrical power is a gigantic part of our datacenter bill (like 60%). However, we cannot really ever turn off the hard drives.

If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back.

> is every pod constantly in use?

The way we built it, yes. If one user deletes a backup or unsubscribes from the Backblaze service, it frees a little bit of space on pods all over the datacenter. So we allow some other customer to use that space up as soon as we can.

We also use ALL the pods in the datacenter to give us "breadth" to accept data faster. This is particularly helpful on surges of new customers due to a press event, or if a competitor like CrashPlan decides to exit the market.

Re: Backblaze has ordered 100 petabytes of hard drives

#72

Earlier quoted context omitted.

Why would it take any time at all to "create a restore"?

Brian from Backblaze here. All the data is encrypted and spread throughout our datacenters. So we fetch each file and decrypt it (with the password you provided when kicking off the restore). If this is less than 1 GByte it is instantaneous. But if it is a TByte of data, it takes a few hours to decrypt and add to the ZIP file.

This is confusing and concerning to me as a potential customer. Having used CrashPlan and other proprietary and FOSS backup software, I don't understand this description of how restoration works with Backblaze.

1. Why is the data being decrypted server-side instead of client-side? The server should be sending the encrypted data to the client, and the password should never leave the client machine.

2. Why are zip files being created? The client software should be receiving the data and writing the files directly to the selected restore directory.

e.g. this is how restore works with CrashPlan, and it only takes a few seconds to begin the process. Files are written directly to the selected output directory.

Re: Backblaze has ordered 100 petabytes of hard drives

#73
post #37

Earlier quoted context omitted.

I don't understand. B2 is currently about half the cost of S3 for outbound data and 1/4 the cost for data at rest. What do you mean "stay competitive?"

They're not equivalent. B2 is singly homed.

So is S3. You have to pick the region (home) for the data when you create the bucket.

Re: Backblaze has ordered 100 petabytes of hard drives

#74
post #27

I imagine you'll have to reduce pricing on Backblaze B2 eventually in order to stay competitive. What's it like having to flip such a massive switch where your revenue is X one minute and then X - 20% the next?

> you'll have to reduce pricing on Backblaze B2 eventually

Yes. We have already dropped B2 prices once, and the expectation is we'll probably be dropping them regularly as our cost per GByte drops.

> What's it like having to flip such a massive switch...

What's amusing is the switch is tiny (just one constant in a json table) and but the fallout is massive. But realistically we have a business strategy of not making egregious profits, so we only drop prices when our margin is too high.

Jeff Bezos of Amazon once said, "Your margin is my opportunity." Well, Jeff's margin is Backblaze's opportunity. :-) We survive and thrive because Amazon is being too greedy and marking up their storage (and outbound bandwidth) by way too much.

Re: Backblaze has ordered 100 petabytes of hard drives

#75
post #63
post #47

Earlier quoted context omitted.

Yev from Backblaze here -> That's definitely not normal, how much data/files do you have (are you doing a .zip or a USB restore)? We're currently QAing a dramatic increase to the restores, but right now only available internally until we are confident in it. In the mean time though you can download mission-critical files in smaller-sized restored, that might help you get up and running faster while the mega-restore c…

Do you reckon the increase in restores has anything to do with APFS? I also lost my computer and Time Machine this week after upgrading to High Sierra, had to restore from backblaze. Btw, unlike parent comment, I had a great experience with the restore and couldn't be happier with the decision to go with backblaze.

HE means they're QA testing a new method of restoring that will dramatically increase restore speed, not that there's a sudden uptick in restores. It's unclearly written, I had to read it twice before I got it.

Re: Backblaze has ordered 100 petabytes of hard drives

#76
post #49
post #35

Earlier quoted context omitted.

Still? 100PB of HDD is still not big Enough? So the question would be, In unit of Backblaze, what are the size for Amazon, Apple, Google, Facebook?

Curious on that too. I wouldn't be surprised if Amazon needs 10,000 drives a day (many of which are probably going to replace dead drives, or upgrade lower capacity ones). Google is probably about the same.

Early on, we (I work at Backblaze) were told by the large drive manufacturers that to do a "direct order" we had to buy at least 10,000 drives in a single order.

If those are 12 TByte Seagates, that's about 120 PBytes? So at this point we are almost there and in fact we sometimes talk directly with the manufacturers, who turn around and send our order through a distributor or reseller. This is probably to prevent "channel conflict" where Seagate or Western Digital is undercutting it's distributors, but that's just a guess.

Re: Backblaze has ordered 100 petabytes of hard drives

#77
post #37
post #27

I imagine you'll have to reduce pricing on Backblaze B2 eventually in order to stay competitive. What's it like having to flip such a massive switch where your revenue is X one minute and then X - 20% the next?

I don't understand. B2 is currently about half the cost of S3 for outbound data and 1/4 the cost for data at rest. What do you mean "stay competitive?"

I mean that prices in the cloud space tend to drop regularly, they can't stay at this price point forever.

Re: Backblaze has ordered 100 petabytes of hard drives

#79

I love the articles from Backblaze. I enjoy them publishing the failure rates of drives and anything else on the blog. Two things missing from them in my opinion 1) a Linux client and 2) a second location I guess I could get away without a Linux client if there was an unRAID plugin for B2.

In previous comments they've indicated that a Linux client won't be released because they can't afford unlimited storage for all the Linux NAS boxes.

B2 is priced at a sustainable point for them, so that's your best option for other use cases-- it's only $5/TB*mo!

Re: Backblaze has ordered 100 petabytes of hard drives

#80

Earlier quoted context omitted.

Brian from Backblaze here. All the data is encrypted and spread throughout our datacenters. So we fetch each file and decrypt it (with the password you provided when kicking off the restore). If this is less than 1 GByte it is instantaneous. But if it is a TByte of data, it takes a few hours to decrypt and add to the ZIP file.

This is confusing and concerning to me as a potential customer. Having used CrashPlan and other proprietary and FOSS backup software, I don't understand this description of how restoration works with Backblaze. 1. Why is the data being decrypted server-side instead of client-side? The server should be sending the encrypted data to the client, and the password should never leave the client machine. 2. Why are zip file…

I'll answer these in reverse order:

> 2. Why are zip files being created?

We actually offer two forms of restore: A) Zip File Download, and B) External USB Hard Drive FedEx'ed to your home.

In the case of the Zip file download, we chose the format of zip because both Mac and PC (the most common desktops) natively understand it with no additional software needed. In other words, if you just lost your computer, go to ANY COMPUTER ANYWHERE and you can fetch your files with a web browser. Zip preserves the file hierarchy and the last modified time, so it's "pretty good".

In the case of the USB Hard Drive there is no zip file, we can correctly place each file with all the correct timestamps in the correct heirarchy on the USB Hard Drive (which is actually an encrypted hard drive) and then FedEx the hard drive to you anywhere in the world in a day or two.

> 1. Why is the data being decrypted server-side instead of client-side?

Short Answer: Ease of use.

Longer Answer: To clarify, Backblaze produces four different products/modes for different customers with different needs and requirements. We want customers to choose what is appropriate for them. One size does not fit all:

1) Online Backup ($5/month) where every file is encrypted on your laptop BEFORE being sent to Backblaze and your backup is secured by your username/password - where you can recover your password if you have access to your email account. (We support two-factor auth which provides an additional optional layer of protection.)

2) Online Backup ($5/month) where every file is encrypted on your laptop BEFORE being sent to Backblaze and your backup is secured by your username/password AND your private encryption key is secured by a "passphrase" that is not recoverable in any way, shape, or form. (Two-factor auth is also optional here.)

3) B2 Object storage (half of 1 cent/GByte/month) where you store your file completely unencrypted, and this can be "private" (only accessible by username/password) or "totally public accessible by knowing the URL". A good application of this is serving up a web page to the public - you really WANT people to see all the contents!

4) B2 Object storage (half of 1 cent/GByte/month) where Backblaze has zero knowledge. You cannot browse your file hierarchy because Backblaze doesn't know your filenames. You cannot preview your images. You cannot recover your passwords. There is no other option other than downloading the encrypted blobs and applying whatever decryption algorithm you decided on (we have no ability to know what that is).

Ok, so I think some (many?) people in the security field think that Backblaze should ONLY offer mode #4 (and maybe #3 to serve up public websites). I happen to disagree and I personally feel that products #1 and #2 are useful and appropriate for some customers. But everybody is welcome to their opinion and we want to be completely open as to what exactly is occurring and what we are offering as a service.

Personally I think #2 is an excellent trade off of security vs convenience. Your data is as impervious to attack as a zero knowledge system in #4 for years upon years. Then one day your laptop is stolen or crashes and you want your files back. You want all 4 TBytes back - so you order one of our free (encrypted) USB hard drives to be FedEx'ed to your home with all your data. To kick this process off FOR THE FIRST TIME EVER you tell us your passphrase (up until this very moment it really has been zero knowledge). At this moment you are opening a window of SLIGHTLY lowered security that slams shut after a few hours. For those few hours of preparing your 4 TByte restore, if an undetected hacker had compromised the one restore server in the Backblaze data center that your job was on, that hacker could possibly get access to your files. But then the reduced security window slams shut, we NEVER write your passphrase to any disk so it has now vaporized and we do not remember it, and if a hacker hacks into our system the following day you are STILL completely impervious.

But again, as long as you fully understand the implications of #3 I am COMPLETELY supportive if you instead choose #4 which is our "Zero Knowledge" offering.

Post reply on HN