Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze has ordered 100 petabytes of hard drives

backblaze.com

Post reply on HN
Print view View original

61–70 of 146 posts

Re: Backblaze has ordered 100 petabytes of hard drives

#61

Post by brianwski »

buro9 wrote: post #18
votepaunchy wrote: post #10

Earlier quoted context omitted.

$50/month at $0.005/GB/month. https://www.backblaze.com/b2/cloud-storage-pricing.html

Aha! I thought B2 was like S3 and figured that Business Backup was what I needed. TIL.

Brian from Backblaze here.

> I thought B2 was like S3

It is. Backblaze actually has two separate product lines:

1) Online Backup for $5/month/computer where you have to use our client we wrote with all of it's policies. This is widely targeted at "ease of use" type customers. There is a free feature inside of "Online Backup" called "Groups" where one person can pay for many others. You can use this for a business, we absolutely do NOT charge differently just because it is a business. Or you can use "Groups" to pay for a family, or whatever. "Groups" is free, we honestly don't care if you use it or not.

2) Backblaze B2 - this is "object storage for half a penny per GByte per month". There are MANY clients available for you to choose from, you can see a list on this page: https://www.backblaze.com/b2/integrations.html The idea here was many people kept approaching Backblaze saying they needed to store stuff cheaply, but didn't want to use our client, or maybe it wasn't "backup" they needed but to host live images on a website on the internet, stuff like that. So we built an API anybody can program to. https://www.backblaze.com/b2/docs/

Re: Backblaze has ordered 100 petabytes of hard drives

#62

Post by atYevP »

linsomniac wrote: post #32

I've watched Backblaze closely, and I love their detailed reports of drive stats. I've always been curious about their choices of manufacturers though. In my past I found that Hitachi/HGST drives were far more reliable than others. We had a couple people doing hardware and software support on ~120 machines, and hard drive replacements were expensive (in man-hours) for us, so whenever possible we got Hitachi drives. T…

Yev from Backblaze here -> We have relationships with manufacturers but don't directly buy from them (yet - fingers-crossed). It really is a combination of price/reliability. At our scale we can afford slightly higher failure rates if it means less expensive drives, so our #1 data point for purchases is the cost per gigabyte. After that we do look at failure rates, but unless it's something wildly out of "normal" we tend to live with the occasional failures. Thanks to Vaults the back-end is designed in such a way that if a drive fails we don't have to run and replace it - so individual failures tend not to stress us out too much anymore!

Re: Backblaze has ordered 100 petabytes of hard drives

#63

Post by biomene »

atYevP wrote: post #47
godzillabrennus wrote: post #34

I’ve been a customer for years and recently had a catastrophic failure of a computer and it’s direct attached backup drive. I have spent the last four days waiting for backblaze to create a restore a backup for a computer on and last I checked it was at 9%. I chatted with support and they said this is normal. That progress may jump to completed any time because the restore only tracks file count. I wish I could say I…

Yev from Backblaze here -> That's definitely not normal, how much data/files do you have (are you doing a .zip or a USB restore)? We're currently QAing a dramatic increase to the restores, but right now only available internally until we are confident in it. In the mean time though you can download mission-critical files in smaller-sized restored, that might help you get up and running faster while the mega-restore c…

Do you reckon the increase in restores has anything to do with APFS? I also lost my computer and Time Machine this week after upgrading to High Sierra, had to restore from backblaze.

Btw, unlike parent comment, I had a great experience with the restore and couldn't be happier with the decision to go with backblaze.

Re: Backblaze has ordered 100 petabytes of hard drives

#64

Post by atYevP »

chrisper wrote: post #59
brianwski wrote: post #57

Earlier quoted context omitted.

Not Yev but I also work at Backblaze. Most of the anti-vibration tech is straight forward, like the fans aren't bolted to the chassis, they are held by rubber rivets, and we have slowly but steadily improved the "pressure" that drives are held in place to limit failures, etc. From time to time, a fan will "half fail" causing large vibrations which can massively increase drive failures in the rack that bad fan is in.

Thanks. Another question I got for you is: What is louder? The fans or all the disks together? I had a workstation with 6 spinning disks and that was already quite loud.

Now it's Yev - you'd have to ask our DC techs. Every time I go to the datacenter it "feels" like the loudest part are the fans, but my ears aren't attuned to what is causing which ruckus :P

Re: Backblaze has ordered 100 petabytes of hard drives

#65

Post by x4d66a5ce »

ac29 wrote: post #46
corobo wrote: post #31

Earlier quoted context omitted.

Is Linux backup support anywhere on the radar at all? I recently trimmed down my Backblaze licenses after switching to Linux installs, just went with s3 as s3cmd had a path of least resistance to storing backups. Would love to switch back if it's ever possible :)

Theyve repeatedly said no -- it breaks their business model to allow Linux machines (often servers) to backup "unlimited" data for $5/month. People with multi-hundred TB NAS machines would end that business model, quickly (see Amazon Cloud Drive, Crashplan, and others). But if youre using S3, just switch to B2, its per GB pricing, much cheaper than S3 and supports lots of Linux tools. I use rclone.

I took the GP's remark about how he "went with s3 as s3cmd had a path of least resistance to storing backups" to mean that he considered B2 but opted for S3 since the latter offers a backup utility, but the former supports neither a "native" (e.g., git or rsync over SSH) backup interface, nor does it have a Linux client—the approach for backup on B2 from desktop Linux is to roll your own backup solution or use somebody else's client that has already added support for the B2 API.

I'm pretty sure the right Linux backup solution is rsync.net, by the way, which does support a native (SSH) file access interface. For big backups, storage is (slightly less than) twice as expensive as S3, but unlike S3, there are no bandwidth costs.

Re: Backblaze has ordered 100 petabytes of hard drives

#66

Post by brianwski »

thebiglebrewski wrote: post #12

Love these articles! Thanks for writing it. Must feel so good to buy that many HDDs at once...something really gratifying about that.

As the guy who signs the check, "gratifying" isn't quite the right word. :-) Some stress, some amazement that everything got this large.

When we started 10 years ago, we could only afford to rent a quarter of one cabinet at the datacenter at 365 Main Street, San Francisco. We had "right of first refusal" on the rest of the cabinet we occupied. We could pile all of the hard drives ever deployed in that cabinet on my kitchen table.

Now it is semi trucks filled with fork lift pallets of drives. But it just "snuck up on us". Each day it was a little larger, each drive order was just slightly larger than the previous.

Re: Backblaze has ordered 100 petabytes of hard drives

#67

Post by jstrassburg »

ericfrederich wrote: post #20

I love the articles from Backblaze. I enjoy them publishing the failure rates of drives and anything else on the blog. Two things missing from them in my opinion 1) a Linux client and 2) a second location I guess I could get away without a Linux client if there was an unRAID plugin for B2.

Second on the linux client. I have a linux machine with a zfs volume that I've been backing up to crashplan. Since they've announced they're dumping their consumer market I have no cost-effective place to go. Most likely, I'll be more picky about my backups and use S3 life-cycled to Glacier.

Re: Backblaze has ordered 100 petabytes of hard drives

#68

Post by brianwski »

monochromatic wrote: post #56
brianwski wrote: post #54

Earlier quoted context omitted.

Brian from Backblaze here. All the data is encrypted and spread throughout our datacenters. So we fetch each file and decrypt it (with the password you provided when kicking off the restore). If this is less than 1 GByte it is instantaneous. But if it is a TByte of data, it takes a few hours to decrypt and add to the ZIP file.

That makes sense, thanks for the response. (4 days and counting still seems excessive though.)

Definitely seems excessive. We'll look into it.

There are some things that can affect it. We have a "pool" of restore servers that do this task. The oldest ones in the pool are 9 year old computers with slow hard drives. The newest restore servers are built on SSDs and are blazingly fast and have newer processors. So based on which restore server was assigned the task, plus other things like the size of each file can speed or slow the restore.

We are in the middle of a project to speed up all restores. Hilariously we figured out that simply by decommissioning the oldest, saddest restore servers the restores averaged FASTER restore times.

Re: Backblaze has ordered 100 petabytes of hard drives

#69

Post by giobox »

mshook wrote: post #40
ericfrederich wrote: post #20

I love the articles from Backblaze. I enjoy them publishing the failure rates of drives and anything else on the blog. Two things missing from them in my opinion 1) a Linux client and 2) a second location I guess I could get away without a Linux client if there was an unRAID plugin for B2.

Regarding 1): rclone does the job. That's what I use with B2. https://rclone.org/

B2 plus rclone works well, but for many it's potentially significantly more expensive than the standard "all you can eat" $5 a month plan. In my case B2 would workout at roughly 4x the monthly cost and that's before factoring in the fact B2 charges for downloads as well as uploads. An official Linux client would be significantly better value.

Re: Backblaze has ordered 100 petabytes of hard drives

#70

Post by dzdt »

atYevP wrote: post #62
linsomniac wrote: post #32

I've watched Backblaze closely, and I love their detailed reports of drive stats. I've always been curious about their choices of manufacturers though. In my past I found that Hitachi/HGST drives were far more reliable than others. We had a couple people doing hardware and software support on ~120 machines, and hard drive replacements were expensive (in man-hours) for us, so whenever possible we got Hitachi drives. T…

Yev from Backblaze here -> We have relationships with manufacturers but don't directly buy from them (yet - fingers-crossed). It really is a combination of price/reliability. At our scale we can afford slightly higher failure rates if it means less expensive drives, so our #1 data point for purchases is the cost per gigabyte. After that we do look at failure rates, but unless it's something wildly out of "normal" we…

If a 10's of petabytes isn't enough scale to talk to manufacturers directly, I wonder what is!
Post reply on HN