Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Google Cloud Storage Nearline

googlecloudplatform.blogspot.com

Post reply on HN
Print view View original

151–160 of 188 posts

Re: Introducing Google Cloud Storage Nearline

#151

Post by oconnore »

panarky wrote: post #50
fuzzythinker wrote: post #35

Earlier quoted context omitted.

Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo.

Glacier pricing is surprisingly complicated, and the actual cost can be much higher than $0.01 per GB-month if you don't read the fine print. The biggest gotcha is that you can only access 0.17% of the data you've stored in any given day without extra charges. So if you've stored 1000 GB, you can only access 1.7 GB per day for free. The cost for going over your daily "retrieval allowance" can be large, because cost i…

Why does Amazon charge like this? It seems like on the rare occasion that you need to send some person to go grab the tape/disk from storage and bring it online, Amazon would want you to get all the data you need and put it back in storage.

Incentivizing users to bring the data online once a day to trickle it out seems bad for all involved.

Re: Introducing Google Cloud Storage Nearline

#153

Post by beolab1700 »

wyldfire wrote: post #150

The transport for this service is HTTP? I imagine most competitors use that as well, right? How does encoding factor in here? I have to transcode my data to base64 in order to put it in or take it out, I assume? I "know" I'd only get billed for the data stored as the original octet/binary encoding. But what about the egress fees? Encoded or decoded data is the input for the billing?

There is no need to base64 encode the data on the way in or way out. I cannot think of any APIs that use base64 encoded requests or responses.

Re: Introducing Google Cloud Storage Nearline

#154

Post by pbreit »

pbreit wrote: post #99
texthompson wrote: post #82

Earlier quoted context omitted.

Nobody is placing restrictions, but it's understandable if people are skeptical.

Praise be the skeptical! It allows the courageous easy advantages. SLAs are almost worthless. And, really, has Google ever retired something like this (i.e., not an acquisition, not Google Reader, etc)?

ReplyTo aros: The two main criteria I would have are: 1) not an acquisition and 2) important development/developer service. I can't think of any and didn't see any after a cursory look at the list.

I think SLAs are literally worthless since I don't think they encourage even slightly more effort in minimizing whatever issues the customer is concerned about. No one wants servers to go down, no one wants to shut down a service. That a few Google bucks might be on the line would have zero impact.

Re: Introducing Google Cloud Storage Nearline

#156

Post by Guvante »

wyldfire wrote: post #150

The transport for this service is HTTP? I imagine most competitors use that as well, right? How does encoding factor in here? I have to transcode my data to base64 in order to put it in or take it out, I assume? I "know" I'd only get billed for the data stored as the original octet/binary encoding. But what about the egress fees? Encoded or decoded data is the input for the billing?

HTTP can transfer arbitrary binary data just fine. There is no need to encode it.

I don't know which they use but HTTP traditionally uses a `Content Length` or an empty chunk to represent the end of data.

Re: Introducing Google Cloud Storage Nearline

#157

Post by anigbrowl »

weitzj wrote: post #73

Quote: "This is a Beta release of Nearline Storage. This feature is not covered by any SLA or deprecation policy and may be subject to backward-incompatible changes." So should I believe in Google's good-will? I would be fine trying out some services, which are in Google Beta. But my valuable data? They should have a SLA right from the start to gain the user's trust.

You shouldn't be trusting your production data to anything in beta. The whole point of open beta testing is to pool the technical risk. So use it, but don't rely on it. If you can't afford to use it in parallel with some existing system, then don't use it.

Re: Introducing Google Cloud Storage Nearline

#158

Post by NeutronBoy »

ak217 wrote: post #135
AceJohnny2 wrote: post #116

Earlier quoted context omitted.

Thanks for the detailed breakdown, that's really interesting! Sounds like you were bitten by this? :)

Don't know about the OP, but we were bitten by this - badly - after our logic to throttle Glacier retrieval failed. AWS has now added the ability to set a spending limit to avoid runaway retrievals. Nonetheless, from my point of view the ultra-complicated pricing scheme is nothing short of a disaster for Glacier as a product. I think it will continue to seriously impact its uptake, and Google is smart to exploit the…

The use case is people who have a requirement to store data and probably never need to access it, either for reg requirements, or legal requirements, etc. In which case paying a couple of hundred bucks for retrieval is fine, especially if they can access at a high level of granularity

edit: Also as others have pointed out, Nearline is limited in retrieval time as well, so the cost difference isn't nearly as large.

Re: Introducing Google Cloud Storage Nearline

#159

Post by tveita »

oconnore wrote: post #151
panarky wrote: post #50

Earlier quoted context omitted.

Glacier pricing is surprisingly complicated, and the actual cost can be much higher than $0.01 per GB-month if you don't read the fine print. The biggest gotcha is that you can only access 0.17% of the data you've stored in any given day without extra charges. So if you've stored 1000 GB, you can only access 1.7 GB per day for free. The cost for going over your daily "retrieval allowance" can be large, because cost i…

Why does Amazon charge like this? It seems like on the rare occasion that you need to send some person to go grab the tape/disk from storage and bring it online, Amazon would want you to get all the data you need and put it back in storage. Incentivizing users to bring the data online once a day to trickle it out seems bad for all involved.

I think it makes most sense to think of it as paying for access to the tape robot.

If Amazon store your data in a tape archive (I don't know if they do, but they at least seem to have similar constraints), they can only access a small portion of the stored data at a time, so they need to control how often people request data.

They could just rate limit everyone, but this way allows people to pay for priority in an emergency while still discouraging everyday read requests.

The pricing makes more sense if you're a large user with data spanning several tapes than if you're in the single terabyte range, but the low limit still discourages you from making requests causally, which helps them keep their SLA.

If they predict that you'll trickle out your small file, they can just read out everything on the first access and cache it online, so there's no extra trips to the archive for them.

Re: Introducing Google Cloud Storage Nearline

#160

Post by discodave »

tveita wrote: post #159
oconnore wrote: post #151

Earlier quoted context omitted.

Why does Amazon charge like this? It seems like on the rare occasion that you need to send some person to go grab the tape/disk from storage and bring it online, Amazon would want you to get all the data you need and put it back in storage. Incentivizing users to bring the data online once a day to trickle it out seems bad for all involved.

I think it makes most sense to think of it as paying for access to the tape robot. If Amazon store your data in a tape archive (I don't know if they do, but they at least seem to have similar constraints), they can only access a small portion of the stored data at a time, so they need to control how often people request data. They could just rate limit everyone, but this way allows people to pay for priority in an em…

You assume Glacier uses tape storage... But the same principle still applies, they're trying to prevent people clogging up the network so when somebody does need to do an emergency restore there is lots of spare network capacity.
Post reply on HN