Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Google Cloud Storage Nearline

googlecloudplatform.blogspot.com

Post reply on HN
Print view View original

61–70 of 188 posts

Re: Introducing Google Cloud Storage Nearline

#61

Post by manacit »

thrownaway2424 wrote: post #33
manacit wrote: post #31

Earlier quoted context omitted.

There's a few reasons: * Dropbox doesn't guarantee 11 9s like this service does - when I'm backing critical data up, I want to make sure it's _there_. * Dropbox likely wouldn't take kindly to me storing 10PB, whereas that is what this service is designed for. * I've got SLAs and guaranteed speeds with this, Dropbox isn't designed for me to suddenly download 10PB very quickly.

Where are you seeing these durability claims? I can't find them. And what would 12 9s durability even mean? You lose one byte out of every TB?

It's 11, not 12, I misspoke.

Those claims are from glacier: http://aws.amazon.com/glacier/faqs/ which is priced the same per GB of data.

In this case, the durability refers to the loss of data/objects stored per year - if you're sending multiple PBs of data off to Glacier, you want to be able to retrieve them many years later. Even 5 9s would mean that 1 object out of 100,000 is lost every year, which is quite poor.

Re: Introducing Google Cloud Storage Nearline

#62

Post by wongarsu »

keysheath wrote: post #14
nda wrote: post #4

from the documentation: "You should expect 4 MB/s of throughput per TB of data stored as Nearline Storage. This throughput scales linearly with increased storage consumption."

That would mean that if you had about 1TB stored it would take more than 3 days to retrieve it (with an initial 3 second delay before it starts).

And since bandwidth scales linearly with used storage it would presumably always take more than 3 days to retrieve all of your data.

Re: Introducing Google Cloud Storage Nearline

#63

Post by klo544 »

gaul wrote: post #32

I added GCS nearline to my object storage comparison: http://gaul.org/object-store-comparison/

Actually there are not 7 regions for Google Cloud Storage but 3: US, EU, ASIA. You can get more specfic but that's still Alpha: ASIA-EAST1, US-CENTRAL1, US-CENTRAL2, US-EAST1, US-EAST2, US-EAST3, US-WEST1.

Note that there are separate zones for Google Compute Engine: https://cloud.google.com/compute/docs/zones#available

Re: Introducing Google Cloud Storage Nearline

#64

Post by panarky »

keysheath wrote: post #52
omh wrote: post #42

Earlier quoted context omitted.

Glacier have additional fees depending on how fast you retrieve the data. Honestly I'm not sure what they are because I've never had to use it, but my impression is that these fees could be quite high.

At least with Glacier I get the option of retrieving faster. Nearline is going to take more than 3 DAYS to retrieve my data if I only have 1TB stored (3MB/s).

Let's assume that with Glacier you can retrieve your 1000 GB in 4 hours instead of 3 days.

You will be surprised by a retrieval fee of 1000 / 4 * 720 * 0.01 = $1,800.

If you want to get your 1000 GB for free, you'd have to request no more than 69 MB per hour. It will take you about 20 months to get your data!

With Glacier, limit retrievals to less than 5% per month / 30 days / 24 hours = 0.007% of total data stored per hour to avoid these ugly surprises.

Source: http://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retr...

Re: Introducing Google Cloud Storage Nearline

#65

Post by moe »

keysheath wrote: post #52
omh wrote: post #42

Earlier quoted context omitted.

Glacier have additional fees depending on how fast you retrieve the data. Honestly I'm not sure what they are because I've never had to use it, but my impression is that these fees could be quite high.

At least with Glacier I get the option of retrieving faster. Nearline is going to take more than 3 DAYS to retrieve my data if I only have 1TB stored (3MB/s).

For many people waiting 3 days is more attractive than paying the ~$1000 that it would cost to retrieve it in one day from Glacier.

Re: Introducing Google Cloud Storage Nearline

#66

Post by Twirrim »

omh wrote: post #42
fuzzythinker wrote: post #35

Earlier quoted context omitted.

Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo.

Glacier have additional fees depending on how fast you retrieve the data. Honestly I'm not sure what they are because I've never had to use it, but my impression is that these fees could be quite high.

They can be. It's possible to set a retrieval policy on your bucket either at free tier, or a fixed price point so you can control the cost of your final bill if you want to retrieve faster than the free tier

Re: Introducing Google Cloud Storage Nearline

#67

Post by keysheath »

nda wrote: post #15
bbrazil wrote: post #12

Earlier quoted context omitted.

My guess would be disk drives that are otherwise used for I/O intensive purposes, and where there's tons of unused/wasted space. 4MB/s is only around 4% of the bandwidth of a modern 1TB hard disk.

You can store 1 Pb and get 4000 MB/s

And at 3MB/s per TB it will always take over 3 days to retrieve all of your data. The 2-5 second retrieval latency is irrelevant when you're going to be waiting for 3 days...

Re: Introducing Google Cloud Storage Nearline

#68

Post by sreitshamer »

moe wrote: post #36

This is a nail in the coffin of the SaaS backup industry (Backblaze, RescuePlan, etc.). With storage becoming essentially free the last excuse for not using self-hosted, secure backup tools like Arq[1] disappears. [1] http://www.haystacksoftware.com/arq/

I'm adding support for creating nearline buckets in Arq right now.

Re: Introducing Google Cloud Storage Nearline

#69

Post by moe »

berberous wrote: post #55
moe wrote: post #46

Earlier quoted context omitted.

Well, I would guess the number of people who need to backup 4T is rather small. My thinking is that most people likely max out well below the 0.5T that the same $5 buys you on Google now, making that variant actually cheaper for them than Backblaze. For those people it means paying less per month and getting privacy and better control (data retention!) in return. Should be a no-brainer.

Backblaze lets you add a personal encryption key. I guess they could log that key when you try to decrypt and restore, although I trust they don't. I suppose the NSA probably gets the key when it gets transmitted, but I don't really care. Am I missing something here? If Backblaze's encryption implementation is substantially worse, I may switch.

I suppose the NSA probably gets the key when it gets transmitted [...] Am I missing something here?

I'm not sure if you're being serious or sarcastic. ;)

Anyway, if you don't care who has access to your backups then obviously my argument has no relevance to you.

Re: Introducing Google Cloud Storage Nearline

#70

Post by berberous »

moe wrote: post #69
berberous wrote: post #55

Earlier quoted context omitted.

Backblaze lets you add a personal encryption key. I guess they could log that key when you try to decrypt and restore, although I trust they don't. I suppose the NSA probably gets the key when it gets transmitted, but I don't really care. Am I missing something here? If Backblaze's encryption implementation is substantially worse, I may switch.

I suppose the NSA probably gets the key when it gets transmitted [...] Am I missing something here? I'm not sure if you're being serious or sarcastic. ;) Anyway, if you don't care who has access to your backups then obviously my argument has no relevance to you.

"Well, what do I have to hide?" is a bad argument with respect to whether the NSA should be doing the things it does, but with respect to whether I am going to spend money and effort to hide my photos and documents from them, I think it's a fair argument.

I care very much, however, whether hackers will have access to my files, as they can cause havoc with things like tax returns that the NSA won't (I mean, the government already has my tax returns....).

So I was mainly curious if there was some significant flaw in Backblaze's encryption that should worry me from the perspective of a non-nation state adversary.

Post reply on HN