Google Cloud Storage Nearline graduates to general availability • Hacker News Board

Re: Google Cloud Storage Nearline graduates to general availability

#31

Post by fweespeech » Thu, Jul 23, 2015, 5:06 PM UTC

vonmoltke wrote: post #26

Zaheer wrote: post #23

Earlier quoted context omitted.

Pulling out backup on the site is $.01/GB - am I missing something?

If I am reading the pricing page[1] correctly, you still get hit with the egress charges on top of the $0.01/GB charge to retrieve from Nearline. Thus, for less than 1TB of data retrieval is $0.13/GB ($0.01 Nearline charge + $0.12 egress charge) if you are not in China or Australia. [1] https://cloud.google.com/storage/pricing#storage-pricing

Yeah, that is how I read the pricing page as well and its consistent with how all other Google and Amazon products of this nature work.

The bandwidth cost is simply too high. They really need a "low quality" v. "standard quality" option. Plenty of people sell bandwidth at $.01-$.04 a GB with 99%-99.99% availability

Re: Google Cloud Storage Nearline graduates to general availability

#32

Post by vgt » Thu, Jul 23, 2015, 5:08 PM UTC

magic5227 wrote: post #29

Does anyone know why bandwidth costs for both AWS and Google have remained flat while storage has rapidly approached .01 cents a gb? Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?

(disclosure: am work for Google). Not sure on AWS.

Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition:

- Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler.

- Google Compute Engine is a VPC out of the box, even cross-region. You can carve out your own sub-VPCs easily using firewall rules.

- Traffic from Google to end-user is not being dumped off of Google network as soon as possible. Google carries your traffic as close to the end-user as possible.

Edit: take a look at this as well.. http://googlecloudplatform.blogspot.com/2015/06/A-Look-Insid...

Re: Google Cloud Storage Nearline graduates to general availability

#33

Post by fweespeech » Thu, Jul 23, 2015, 5:14 PM UTC

vonmoltke wrote: post #28

fweespeech wrote: post #17

As cute as it is to have $.01/GB backup storage....every time you pull out a backup it costs you $.13/GB. That is a pretty steep hit when you can use a lower quality object store: https://www.runabove.com/storage/object-storage.xml For $.01/GB and $.01/GB to pull. Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 1…

However, the egress charge only applies to data moved off Google Cloud Services. If you use a Google compute instance to verify the backup (same size as the Linode at $0.05/hr), the cost becomes: (7 * 50 * .01) + (30 * .05) + (30 * .01 * 50) = $20.00/month

You forgot disk pricing for apples to apples. You also lose multi-provider redundancy [e.g. Everything must be on Google cloud or you are maintaining multiple builds of your production database machines ]

https://cloud.google.com/compute/pricing#localssdpricing

$0.113

So its:

[apples to apples; local ssd]

$186.95 = (7 * 50 * .01) + (30 * .05) + (30 * .1213 * 50)

[persistent provisioned ssd]

$54 = (7 * 50 * .01) + (30 * .05) + ( ( ( (.17 * 96) / 720) + .01) * 50 * 30)

Re: Google Cloud Storage Nearline graduates to general availability

#34

Post by bbrazil » Thu, Jul 23, 2015, 5:16 PM UTC

xhrpost wrote: post #20

Are there any well accepted theories as to how these (Nearline & Glacier) systems function? I know with Glacier, I've heard powering down hard-drives, advanced compression (explains 3hr retrieval), and blu-ray storage (explains 3 month minimum). With Nearline bringing access down to 3secs for the same cost, and now with On-Demand IO, I really don't know what to speculate. Assuming I/O is limited due to spread across…

My conjecture on Nearline is that it's unused disk space on disks that are mostly I/O bound.

Re: Google Cloud Storage Nearline graduates to general availability

#35

Post by willcodeforfoo » Thu, Jul 23, 2015, 5:18 PM UTC

Any tips for getting a fairly significant amount of data (50TB) from a local NAS to Google quicker than a 5mbps upstream will allow?

Amazon has Import/Export which lets you ship drives, is this the best option?

Re: Google Cloud Storage Nearline graduates to general availability

#36

Post by boundlessdreamz » Thu, Jul 23, 2015, 5:54 PM UTC

vgt wrote: post #32

magic5227 wrote: post #29

Does anyone know why bandwidth costs for both AWS and Google have remained flat while storage has rapidly approached .01 cents a gb? Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?

(disclosure: am work for Google). Not sure on AWS. Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition: - Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler. - Google Compute Engine is a VPC…

But you are much more expensive than S3 & CloudFront when it comes to request pricing

Cloud Storage considers retrieving an object through its HTTP url is considered a class B XML request type which is priced at $0.01/10,000 ops. This is 1.3x-2.5x more expensive than cloudfront and s3 respectively. This is also not well documented and caused me some trouble. I didn't expect HTTP GET requests to be counted as XML API requests.

Re: Google Cloud Storage Nearline graduates to general availability

#37

Post by edutechnion » Thu, Jul 23, 2015, 6:05 PM UTC

magic5227 wrote: post #29

Does anyone know why bandwidth costs for both AWS and Google have remained flat while storage has rapidly approached .01 cents a gb? Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?

From 2014-11:

“Networking is a red alert situation for us right now,” explained Hamilton. “The cost of networking is escalating relative to the cost of all other equipment. It is Anti-Moore. All of our gear is going down in cost, and we are dropping prices, and networking is going the wrong way. That is a super-big problem, and I like to look out a few years, and I am seeing that the size of the networking problem is getting worse constantly. At the same time that networking is going Anti-Moore, the ratio of networking to compute is going up.”

http://www.enterprisetech.com/2014/11/14/rare-peek-massive-s...

Re: Google Cloud Storage Nearline graduates to general availability

#38

Post by skystorm » Thu, Jul 23, 2015, 6:06 PM UTC

boundlessdreamz wrote: post #36

vgt wrote: post #32

Earlier quoted context omitted.

(disclosure: am work for Google). Not sure on AWS. Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition: - Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler. - Google Compute Engine is a VPC…

But you are much more expensive than S3 & CloudFront when it comes to request pricing Cloud Storage considers retrieving an object through its HTTP url is considered a class B XML request type which is priced at $0.01/10,000 ops. This is 1.3x-2.5x more expensive than cloudfront and s3 respectively. This is also not well documented and caused me some trouble. I didn't expect HTTP GET requests to be counted as XML API…

What did you expect GET requests to be counted as then? As an aside, I think it's fairly well documented: https://cloud.google.com/storage/pricing#operations-pricing (lists "GET Object" under class B) [but disclosure: I work for Google]

Re: Google Cloud Storage Nearline graduates to general availability

#39

Post by pa7ch » Thu, Jul 23, 2015, 6:06 PM UTC

vgt wrote: post #32

magic5227 wrote: post #29

Does anyone know why bandwidth costs for both AWS and Google have remained flat while storage has rapidly approached .01 cents a gb? Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?

(disclosure: am work for Google). Not sure on AWS. Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition: - Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler. - Google Compute Engine is a VPC…

"SSL added and removed here! :¬)"

Re: Google Cloud Storage Nearline graduates to general availability

#40

Post by edutechnion » Thu, Jul 23, 2015, 6:13 PM UTC

willcodeforfoo wrote: post #35

Any tips for getting a fairly significant amount of data (50TB) from a local NAS to Google quicker than a 5mbps upstream will allow? Amazon has Import/Export which lets you ship drives, is this the best option?

So 5mbps is 625KBps or 0.000000625 TB per second.

50 TB / 0.000000625 TBps = 80000000 seconds = 925.92 days

AWS Import can probably import at 100MBps or ~6 days.