Live data from Hacker News

Introducing Google Cloud Storage Nearline

googlecloudplatform.blogspot.com

131–140 of 188 posts

Re: Introducing Google Cloud Storage Nearline

#131
post #36

This is a nail in the coffin of the SaaS backup industry (Backblaze, RescuePlan, etc.). With storage becoming essentially free the last excuse for not using self-hosted, secure backup tools like Arq[1] disappears. [1] http://www.haystacksoftware.com/arq/

I'm adding support for creating nearline buckets in Arq right now.

I just came in here just to ask/comment on: I hope Arq implements this. It's great to see such a prompt response. Thanks

Re: Introducing Google Cloud Storage Nearline

#132
post #38
post #36

This is a nail in the coffin of the SaaS backup industry (Backblaze, RescuePlan, etc.). With storage becoming essentially free the last excuse for not using self-hosted, secure backup tools like Arq[1] disappears. [1] http://www.haystacksoftware.com/arq/

Backblaze is $5/mo per computer. If I have 4TB on a computer, it's still just $5/mo.

"Backblaze is $5/mo per computer. If I have 4TB on a computer, it's still just $5/mo."

Beware flat rate service providers - it's not a consumer-friendly model:

http://blog.kozubik.com/john_kozubik/2009/11/flat-rate-stora...

Your interests and the interests of your provider should be aligned, not opposed.

Re: Introducing Google Cloud Storage Nearline

#133

Earlier quoted context omitted.

How is using Arq+Nearline better than Backlaze (client and storage)? Not really "self hosted" if you're still dumping the data to a cloud provider. Even if you configure Arq to use your own "cloud" (cost of that notwithstanding) I don't think it's an overall net improvement.

(I'm the developer behind Arq) With Arq you keep control of your stuff because it's encrypted before it leaves your computer. With Backblaze you can choose your own encryption key, but to restore your files you have to give them your key; they decrypt your files and leave them (unencrypted!) on their servers for you to download, which I think makes choosing your own encryption key pointless. https://news.ycombinator.…

arq (which we love) + rsync.net = success.

We have a "HN readers" discount which is fairly substantial.

Re: Introducing Google Cloud Storage Nearline

#134

Earlier quoted context omitted.

I'm adding support for creating nearline buckets in Arq right now.

This is exactly what I want for Linux (headless), but I haven't been able to find anything like it. The closest thing I've found is attic, which, while amazing, requires a daemon to be running on the remote end (which is what makes checking, deduplication and deltas fast). Does anyone know an Arq alternative for linux?

duplicity ...

https://en.wikipedia.org/wiki/Duplicity_(software)

Re: Introducing Google Cloud Storage Nearline

#135
post #50

Earlier quoted context omitted.

Glacier pricing is surprisingly complicated, and the actual cost can be much higher than $0.01 per GB-month if you don't read the fine print. The biggest gotcha is that you can only access 0.17% of the data you've stored in any given day without extra charges. So if you've stored 1000 GB, you can only access 1.7 GB per day for free. The cost for going over your daily "retrieval allowance" can be large, because cost i…

Thanks for the detailed breakdown, that's really interesting! Sounds like you were bitten by this? :)

Don't know about the OP, but we were bitten by this - badly - after our logic to throttle Glacier retrieval failed.

AWS has now added the ability to set a spending limit to avoid runaway retrievals. Nonetheless, from my point of view the ultra-complicated pricing scheme is nothing short of a disaster for Glacier as a product. I think it will continue to seriously impact its uptake, and Google is smart to exploit the opportunity to offer a simple, straightforward pricing scheme.

Re: Introducing Google Cloud Storage Nearline

#136

Earlier quoted context omitted.

For a reference point: The hardware cost for storage servers with full redundancy is in the order of $0.10 per GB (plus electricity, rack space etc.). Buying your own hardware and throwing it away every year is price competitive with Nearline (and you get full online storage, no three second delays and slow retrieval). Of course there's good reasons not to roll your own solution until you reach a certain scale, but i…

> For a reference point: The hardware cost for storage servers with full redundancy is in the order of $0.10 per GB (plus electricity, rack space etc.). Buying your own hardware and throwing it away every year is price competitive with Nearline (and you get full online storage, no three second delays and slow retrieval). Well, it would be price competitive if you considered only the hardware costs and not, e.g., the…

Consider that there are labor costs around operating services on AWS as well, and it's generally more expensive by the hour, and labor cost of managing your own servers is fairly low.

I have plenty of servers sitting in racks that haven't been touched in 5+ years; if you were to operate on a "buy and throw out in a year" practice, my typical labour costs would average a couple of hours per new server for setup. Or if you rent managed servers, you don't ever need to touch (or see) the hardware.

Conversely, you can get bandwidth at a tiny fraction of the cost of AWS bandwidth, so the moment you actually transfer data to/from your setup, AWS gets progressively more expensive.

Re: Introducing Google Cloud Storage Nearline

#137
post #49

Earlier quoted context omitted.

Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo.

Except for the Glacier retrieval fees which can be insane if you choose to retrieve more than 0.17% of your stored data per day. https://aws.amazon.com/glacier/faqs/#How_much_data_can_I_ret...

But if you retrieve the same 1000 GB from Glacier in the same 3 days that Google takes you would pay a 98 dollar retrieval fee. And Amazon would charge 300 dollar less traffic cost ($ 0.09 vs $ 0.12 / GB for 1000 GB) so in the end Glacier is still 202 dollar cheaper AND allows you to retrieve the data much faster if you really have to at a certain price.

So Google looks cheaper/simpler, but the amazon model is actually quite good for disaster recovery where you may be in a situation where you "need it now at any cost"

Re: Introducing Google Cloud Storage Nearline

#138
post #133

Earlier quoted context omitted.

(I'm the developer behind Arq) With Arq you keep control of your stuff because it's encrypted before it leaves your computer. With Backblaze you can choose your own encryption key, but to restore your files you have to give them your key; they decrypt your files and leave them (unencrypted!) on their servers for you to download, which I think makes choosing your own encryption key pointless. https://news.ycombinator.…

arq (which we love) + rsync.net = success. We have a "HN readers" discount which is fairly substantial.

= success

rsync.net is 20x more expensive than Google Nearline ($0.20/GB).

Why would anyone choose to use it for Arq when Arq supports both?

Re: Introducing Google Cloud Storage Nearline

#140
post #46

Earlier quoted context omitted.

Well, I would guess the number of people who need to backup 4T is rather small. My thinking is that most people likely max out well below the 0.5T that the same $5 buys you on Google now, making that variant actually cheaper for them than Backblaze. For those people it means paying less per month and getting privacy and better control (data retention!) in return. Should be a no-brainer.

Backblaze lets you add a personal encryption key. I guess they could log that key when you try to decrypt and restore, although I trust they don't. I suppose the NSA probably gets the key when it gets transmitted, but I don't really care. Am I missing something here? If Backblaze's encryption implementation is substantially worse, I may switch.

Can anyone comment how Crashplan stacks up vs Backblaze here? I'm leaning towards it as it'll save the data for more than 30 days after deletion, and security's another factor in my choice.
Post reply on HN