Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze B2 Cloud Storage

backblaze.com

Post reply on HN
Print view View original

211–220 of 237 posts

Re: Backblaze B2 Cloud Storage

#211

Post by vidarh »

venomsnake wrote: post #81

> Downloads are just $0.05/GB Can anyone compare that to other similar providers? While the storage is cheap, it seems more useful for cold storage.

It's cheap for blob storage. It's terribly expensive compared to hosting yourself.

So yeah, I'd agree with you. But for anyone prepared to use S3 for anything but cold storage, this is still a lot cheaper.

My suggestion would be to use this for cold storage + big cache boxes at a provider with low bandwidth charges. Especially if your "hot" objects make up a relatively small percentage.

Re: Backblaze B2 Cloud Storage

#212

Post by hga »

jbeda wrote: post #181
devit wrote: post #171

Earlier quoted context omitted.

No, it's pretty much totally unnecessary. The API works on top of TLS, which already includes cryptographic authentication of all data (usually via SHA-1/2 HMAC or AES-GCM). The hash would be computed at the client right after reading from disk and right before TLS enryption, and since they seem to terminate TLS at the storage server it would be computed right after TLS decryption and right before storage, so it does…

When operating at scale, you will, once in a while, have corruption. Even if you use ECC RAM, once in a while you'll have a double bit flip. And it doesn't look like Backblaze uses ECC ( https://www.backblaze.com/blog/storage-pod-4-5-tweaking-a-pr... ) despite good evidence that ECC is necessary (PDF: http://static.googleusercontent.com/media/research.google.co... ). Even if you do have ECC, you'll once in a while ha…

I just spent 10 or so minutes and it looks like they do use ECC, and per https://news.ycombinator.com/item?id=2786695 see ECC corrections reported in their log files.

Re: Backblaze B2 Cloud Storage

#213

Post by ghshephard »

Dylan16807 wrote: post #185
jbeda wrote: post #147

Earlier quoted context omitted.

Using a hash or CRC here is totally necessary. Often times CRCs in TCP fail due to corruption outside the network stack. Having an end to end check will catch, say, memory bit flips and such after data comes off the wire. But there is no call for a cryptographic hash here. This isn't being used as any sort of ID or to verify integrity outside of corruption.

A cryptographic hash is pretty much as fast as anything else, and lets you be certain. There's no good reason to use anything else.

Apparently, SHA-1 is pretty slow compared to others, about 20x slower than the fastest hash algorithms out there.

https://github.com/Cyan4973/xxHash

You would think, that if it's just being used as a checksum, anything that passes https://code.google.com/p/smhasher/wiki/SMHasher with high marks would be sufficient.

Re: Backblaze B2 Cloud Storage

#214

Post by Osmium »

tedunangst wrote: post #163
Osmium wrote: post #120

Earlier quoted context omitted.

> They use Blowfish. Says it all really - their default encryption is a long-obsolete 64-bit block cipher you might have picked in 1999 because it was faster than 3DES. Not defending it, because I know it's old and there are weaknesses, but aren't Blowfish and 3DES both still technically secure? This is a genuine question. It was my understanding that if implemented correctly, with a random key etc., that neither has…

The 64 bit block size is a (minor) problem if you're storing lots of data because you start seeing duplicates.

Doesn't this depend on the mode of operation/block chaining? Or is it an issue regardless?

Re: Backblaze B2 Cloud Storage

#215

Post by johntash »

brianwski wrote: post #152
wcfields wrote: post #132

Earlier quoted context omitted.

Love to see this as well. Part of the selling point of Backblaze as an online backup is the ability to get a FedEx'd hard drive. I could see this being a practicality for small time video editors, I need to keep copies of old projects, but would be willing to pay $300 to get an overnight hard drive of the files since whatever new project will pay for that cost.

Up to the first 4 TBytes it is only $189 which includes you keeping the hard drive. I'll make it even better - if you ship us back the hard drive within 30 days (you pay return shipping) we'll refund the entire $189. The info on the program is published here: https://help.backblaze.com/entries/67512970-How-to-order-a-r...

Thanks for the information, I didn't know those drives were refundable! I was always turned off by having to download a huge zip file from backblaze when I wanted to restore something, but this actually sounds really nice of you guys for large datasets.

Re: Backblaze B2 Cloud Storage

#216

Post by ck2 »

PhantomGremlin wrote: post #156
ck2 wrote: post #20

Earlier quoted context omitted.

Tigerdirect had them for $170 for a few days. Note TDs are air-filled 7200rpm regular drives. Are those 8TB and 10TB the helium ones with HAMR? Very slow.

Are those 8TB and 10TB the helium ones with HAMR? Very slow. My understanding of HAMR is that it is probably perfectly fine for Backblaze's backup products, which are (more or less) write-once, read-rarely. Shingled magnetic recording should also be OK for that use case. But clearly not good for cloud storage.

Heat assisted is fine for write once but helium filled drives actually leak over time and fail because their molecules are so small.

Re: Backblaze B2 Cloud Storage

#217

Post by hobs »

brianwski wrote: post #177
hobs wrote: post #169

Earlier quoted context omitted.

When I was a customer of BB I noticed no issue with the uploads, but actually when I had a flood and my hardware was destroyed, redownloading all my information was order of magnitudes slower. I tried from multiple physical locations but I could not increase my downloads past 1-2mbps, and for TB of data, that seemed like it was throttled by BB considering I was easily uploading 20mbps. I contacted BB support and they…

Brian from Backblaze here. I wonder if that was during the incredibly annoying "Comcast goes to war with Netflix" era that Backblaze got caught up in. That was Nov 2013 through Feb 2014, you can read a little about it here: https://www.backblaze.com/blog/obama-backs-net-neutrality/ (scroll down for our graphs showing our customers getting throttled). That seriously sucked for Backblaze. But either way, we added threa…

Brian, thank you for responding, I appreciate your clarity and honesty.

My issue did occur during that period, and I am impressed you can call that out from memory, it must have been a frustrating time for BB.

If that alone was the problem, you would have just 100% won back a customer, but the thing that irked me the most was the customer support response.

I know you do not work for your helpdesk, but their response was more the reason I left, their apparent lack of concern was what turned one of any service provider malfunctions into a dissatisfied customer looking for a competitor.

Re: Backblaze B2 Cloud Storage

#218

Post by Dylan16807 »

ghshephard wrote: post #213
Dylan16807 wrote: post #185

Earlier quoted context omitted.

A cryptographic hash is pretty much as fast as anything else, and lets you be certain. There's no good reason to use anything else.

Apparently, SHA-1 is pretty slow compared to others, about 20x slower than the fastest hash algorithms out there. https://github.com/Cyan4973/xxHash You would think, that if it's just being used as a checksum, anything that passes https://code.google.com/p/smhasher/wiki/SMHasher with high marks would be sufficient.

Why would you want 'just' a checksum? I want something I can rely on. If I have to dedicate half a core per gbps of internet-crossing upload, that's not a big deal.

Re: Backblaze B2 Cloud Storage

#219

Post by 2bluesc »

voltagex_ wrote: post #196
brianwski wrote: post #167

Earlier quoted context omitted.

A python command line tool that should work on Linux to send folders to B2 is available for download here: https://www.backblaze.com/b2/docs/quick_command_line.html CAVEAT (PLEASE READ): this does NOT encrypt data yet!! This is just a quick technology demonstration, it isn't a polished backup client. Give us another month for that... We expect Linux servers (and desktops) to make up a significant percentage of the th…

If I was going to tar and then gpg files, what would be the optimal "chunk size" from your point of view? Say I have 10GB that I'm encrypting and then splitting up, are you going to want 1GB chunks or many more 100mb chunks? (This also raises the question of how much data I want to lose to corruption...)

One thing to consider is using ecryptfs as I do for my data at rest. It's mounted as a (mostly) regular filesystem at ~/Private (or similar) and the encrypted files that serve as the backend (~/.private) are then uploaded to the cloud service of my choice. Currently using crashplan, but B2 would be nice to create archives/snapshots of my data instead of just syncing it as most consumer backup solutions do.

This is a very low friction way to have accessible data and have it encrypted by a fairly popular mechanism.

Re: Backblaze B2 Cloud Storage

#220

Post by derefr »

brianwski wrote: post #184
derefr wrote: post #160

Earlier quoted context omitted.

> If your programmer makes a mistake and a line of code deletes some mission critical data from Amazon S3, then all the Reed-Solomon encoding in the world doesn't help you, the data is still gone. I'm surprised at the implication here, that you'd use Glacier on a non-versioned bucket. Making destructive updates impossible doesn't cost much extra in archive fees.

Ok, so let's say you forget to pay your Glacier bill because the IT guy left and the credit card changed and the alert emails go nowhere. Bye-Bye-Glacier! No payment, no customer data, Amazon might delete your data due to a tiny administrative screwup. My point stands: if you don't mind losing your data, store it in one vendor. But if you would REALLY lose your business and put 10 people out of work if the data is lo…

But... let's say you have three accounts with three storage companies. Now let's say you outsource the management of those accounts through one company... or even, equivalently, delegate it to a subsidiary or partner of your company. And then you accidentally stop paying them, or they can't requisition the budget necessary to pay the providers, or whatever. Now you're still stuck, even though you're nominally doing things "in-house."

What you actually need is a provider that will guarantee the durability of your data even if they (temporarily) cut off your access to it for lack of payment[1]. Anything else is just a level of indirection that suffers the same problems.

---

[1] I don't actually know if anyone does this, let alone AWS. Here's a quote from Tarsnap's FAQ—where you'd think cperciva is someone who would have considered the "I had no idea my infrastructure was relying on this service until it shut off" use-case:

> You will be sent an email when your account balance falls below 7 days worth of storage costs warning you that you should probably add more money to your account soon. If your account balance falls below zero, you will lose access to Tarsnap, an email will be sent to inform you of this, and a 7 day countdown will start; if your account balance is still below zero after 7 days, it may be deleted (along with any data you have stored) at our discretion. (If you can't add money yet but will be able to later, contact us and explain the situation. We're reasonable people and simply knowing that you're alive and haven't forgotten that you were using Tarsnap is very helpful.)

7 days is probably reasonable in the case where there's an active IT staff who will notice when, say, servers stop backing up. But if nobody's watching for that...

Post reply on HN