Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze has ordered 100 petabytes of hard drives

backblaze.com

Post reply on HN
Print view View original

131–140 of 146 posts

Re: Backblaze has ordered 100 petabytes of hard drives

#131

Post by atYevP »

rasz wrote: post #128
atYevP wrote: post #19

Earlier quoted context omitted.

Yev from Backblaze here -> Most of our purchasing is done through distributors. We'd love to buy direct at some point, but for now we have good manufacturer/distributor relationships! We're getting there though!

There is a simple fix - become a distributor. I worked for local European distributors that moved less merchandise per year and had direct relationships with Samsung/Fujitsu/IBM/WD/Seagate (you can tell it was a long time ago :P).

We've batted the idea around :P

Re: Backblaze has ordered 100 petabytes of hard drives

#132

Post by brianwski »

explainplease wrote: post #110
brianwski wrote: post #80

Earlier quoted context omitted.

I'll answer these in reverse order: > 2. Why are zip files being created? We actually offer two forms of restore: A) Zip File Download, and B) External USB Hard Drive FedEx'ed to your home. In the case of the Zip file download, we chose the format of zip because both Mac and PC (the most common desktops) natively understand it with no additional software needed. In other words, if you just lost your computer, go to A…

Thanks for your detailed answer! > In the case of the Zip file download, we chose the format of zip because both Mac and PC (the most common desktops) natively understand it with no additional software needed. In other words, if you just lost your computer, go to ANY COMPUTER ANYWHERE and you can fetch your files with a web browser. Zip preserves the file hierarchy and the last modified time, so it's "pretty good". S…

> it takes time and space for you to prepare the zip file,

> and space and time for me to save it and extract it

One advantage is that for millions of small files, batching them together into one monolithic ZIP file allows for faster downloading. As opposed to a round trip to fetch each file. But there could be a happy medium where we batch 1,000 encrypted files to download to your computer then split them up, decrypt them, and cleanup any temporary files before fetching the next 1,000 encrypted files, etc.

> Is there any chance of implementing a hybrid of #2 and #4,

> a client-based, zero-knowledge option?

Yeah, we get asked for a "client restore option" quite a bit (see other comments in this thread also). Realistically I don't see it happening in the next 6 months, but eventually we really should get it done.

Re: Backblaze has ordered 100 petabytes of hard drives

#133

Post by brianwski »

elfchief wrote: post #89
brianwski wrote: post #80

Earlier quoted context omitted.

I'll answer these in reverse order: > 2. Why are zip files being created? We actually offer two forms of restore: A) Zip File Download, and B) External USB Hard Drive FedEx'ed to your home. In the case of the Zip file download, we chose the format of zip because both Mac and PC (the most common desktops) natively understand it with no additional software needed. In other words, if you just lost your computer, go to A…

I would really like #2 to have the option of sending encrypted files on disk or zip, and y'all providing a client that will decrypt them using my passphrase, locally. Nothing personal, but I really don't want y'all seeing my data, even if I need to restore. (context: I've been a Backblaze user for years and love the hell out of you. I've never really thought of how the restore would work with my encrypted data, thoug…

> Nothing personal, but I really don't want y'all seeing my data, even if I need to restore.

We totally understand, especially about the (very real) security issues.

I would reassure you that Zip file restores are COMPLETELY automated and with thousands of them happening every day no Backblaze employee ever sees your files.

The USB restore drives are placed in an envelope by a human, but they are encrypted when they are placed in the envelope so there is still no danger of any person seeing your files.

Plus we take our customer's privacy VERY seriously and it would be a firing offense for any Backblaze employee to ever look at a single customer file (or file name) without that customer's explicit permission.

Re: Backblaze has ordered 100 petabytes of hard drives

#134

Post by jlgaddis »

godzillabrennus wrote: post #91
woolvalley wrote: post #87

Earlier quoted context omitted.

This is why you always want the onsite & offsite backup, because offsite cloud backups will always be fairly slow to fully restore. I use a local NAS to achieve automatic onsite backups myself. It's part of the 3-2-1 backup rule.

I had a Mac Pro Mid-2010 workstation with a RAID 5 Array configured to have a hot spare for redundancy. Then I had a locally attached hard drive for time machine backups + BackBlaze + a hosted repo for the work I do + I use a third party encrypted cloud file sharing service for the most important documents. The problem started when I decided to upgrade to High Sierra. The upgrade was catastrophic for this configurati…

It sucks that you have to deal with this (I've been there) but I appreciate the added detail. Along with a few other stories I've heard, this has me leery of upgrading to High Sierra in the near future (although my MBP doesn't really have anything of importance on it).

Good luck, hopefully you can get everything back in order before the weekend is over.

Re: Backblaze has ordered 100 petabytes of hard drives

#135

Post by [deleted] »

e40 wrote: post #94
godzillabrennus wrote: post #91

Earlier quoted context omitted.

I had a Mac Pro Mid-2010 workstation with a RAID 5 Array configured to have a hot spare for redundancy. Then I had a locally attached hard drive for time machine backups + BackBlaze + a hosted repo for the work I do + I use a third party encrypted cloud file sharing service for the most important documents. The problem started when I decided to upgrade to High Sierra. The upgrade was catastrophic for this configurati…

For the life of me, I can't figure out why people upgrade to new operating systems so quickly. I never upgrade my macOS until x.x.4, but I prefer x.x.5. I'm on 10.11.6 because 1) it works perfectly, and 2) Karabiner doesn't work on 10.12.

[deleted]

Re: Backblaze has ordered 100 petabytes of hard drives

#136

Post by amelius »

sml156 wrote: post #127
amelius wrote: post #85

Earlier quoted context omitted.

> If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back. Well, speaking for myself, I don't mind if I have to restore a backup, and have to wait 1 minute extra for a system to come online (that's about the typical boot time of a Linux system, so I suppose for a storage pod it would be about the same). By the way, it…

Interesting, your considering that your setup somehow compares to Blackblaze Also if if he says it takes about 15 Min's for it to come online and be usable in their massive data center I would believe him.

Not really, he was talking about Amazon's solution.

Re: Backblaze has ordered 100 petabytes of hard drives

#137

Post by jlgaddis »

jstrassburg wrote: post #67
ericfrederich wrote: post #20

I love the articles from Backblaze. I enjoy them publishing the failure rates of drives and anything else on the blog. Two things missing from them in my opinion 1) a Linux client and 2) a second location I guess I could get away without a Linux client if there was an unRAID plugin for B2.

Second on the linux client. I have a linux machine with a zfs volume that I've been backing up to crashplan. Since they've announced they're dumping their consumer market I have no cost-effective place to go. Most likely, I'll be more picky about my backups and use S3 life-cycled to Glacier.

I would really like to use rsync.net for my "external third-party" backups specifically because of their "ZFS Platform" [0] but (from my off the top of my head calculations) their pricing seems to come out a little on the high side.

With my luck, the first time I need to be able to quickly restore a full dataset or something, it will probably have been well worth it and I'll likely be kicking myself for not doing it.

[0]: http://www.rsync.net/products/platform.html#zfs

Re: Backblaze has ordered 100 petabytes of hard drives

#138

Post by jlgaddis »

giobox wrote: post #96
086421357909764 wrote: post #90

Earlier quoted context omitted.

As they've said though, the usage of most linux users isn't on par with Windows / Mac users. They simply would crush themselves with people attempting to backup 200TB nas devices for $5.00 a month. If you want any perspective on how crazy the fringes are with data storage and how they impact providers (such as amazon, google, backblaze, etc.) simply read the stories on reddit /r/datahoarder and you'll quickly see why…

Very small numbers of people already do this on Mac/Windows using tricks like mouting the NAS as an iSCSI device to backup absurd amounts on the $5 plan. The business model is clearly that the legions of small users subsidize the "whales" and Linux desktops are a drop in the ocean vs the Windows/Mac world too. Do you have a source for where Backblaze claimed the usage of most Linux users isn't on par?

> Do you have a source for where Backblaze claimed the usage of most Linux users isn't on par?

One of their employees pretty much said exactly that here on HN recently. I don't have the comment bookmarked or anything like that so I can't cite it for you, but I do remember reading it.

I'm paraphrasing, of course, but s/he basically said "the math doesn't work out" (for the regular service, not B2 (obviously)).

Re: Backblaze has ordered 100 petabytes of hard drives

#139

Post by marktangotango »

brianwski wrote: post #74
xhrpost wrote: post #27

I imagine you'll have to reduce pricing on Backblaze B2 eventually in order to stay competitive. What's it like having to flip such a massive switch where your revenue is X one minute and then X - 20% the next?

> you'll have to reduce pricing on Backblaze B2 eventually Yes. We have already dropped B2 prices once, and the expectation is we'll probably be dropping them regularly as our cost per GByte drops. > What's it like having to flip such a massive switch... What's amusing is the switch is tiny (just one constant in a json table) and but the fallout is massive. But realistically we have a business strategy of not making…

This, my friends, is the way it's done. Kudos!

Re: Backblaze has ordered 100 petabytes of hard drives

#140

Post by explainplease »

brianwski wrote: post #132
explainplease wrote: post #110

Earlier quoted context omitted.

Thanks for your detailed answer! > In the case of the Zip file download, we chose the format of zip because both Mac and PC (the most common desktops) natively understand it with no additional software needed. In other words, if you just lost your computer, go to ANY COMPUTER ANYWHERE and you can fetch your files with a web browser. Zip preserves the file hierarchy and the last modified time, so it's "pretty good". S…

> it takes time and space for you to prepare the zip file, > and space and time for me to save it and extract it One advantage is that for millions of small files, batching them together into one monolithic ZIP file allows for faster downloading. As opposed to a round trip to fetch each file. But there could be a happy medium where we batch 1,000 encrypted files to download to your computer then split them up, decryp…

> One advantage is that for millions of small files, batching them together into one monolithic ZIP file allows for faster downloading. As opposed to a round trip to fetch each file. But there could be a happy medium where we batch 1,000 encrypted files to download to your computer then split them up, decrypt them, and cleanup any temporary files before fetching the next 1,000 encrypted files, etc.

Well, any such software that didn't have pipelining would be poorly designed. ;) Compare to software like rsync, Unison, etc, which transfer large numbers of small files quickly. Seems like a good solution would be to build on librsync.

Post reply on HN