Live data from Hacker News

Google Cloud Storage Nearline graduates to general availability

googlecloudplatform.blogspot.com

41–50 of 58 posts

Re: Google Cloud Storage Nearline graduates to general availability

#42
post #19

Earlier quoted context omitted.

Once the free period is over the 100PB will cost $1 million per month in storage fees alone. Will the free month(s) even cover the retrieval and transfer fees to take 100PB out of another cloud?

To store 100PB, one can always buy 50.000 hard drives. Would only cost $2-3 million of initial payment. I wonder how much maintenance would cost to provide same availability with near storage.

Don't forget the full-time admins and hwops people. Operations costs are going to dwarf the capital investment.

Re: Google Cloud Storage Nearline graduates to general availability

#43
post #16

Earlier quoted context omitted.

Its a different use case, cold storage vs content delivery...

Not really. Cold storage is absolutely a necessity when you reach the scale of imgur, unless you want to run the service incredibly inefficiently. imgur keeps photos forever [0]. To do so without some kind of cold storage infrastructure in place for all the millions of photos that will probably never be seen again, or perhaps at a rate of once per year, would be infrastructure cost suicide. Facebook faces a similar c…

That's weird, I've seen conflicting information that says imgur deletes images not accessed for 180 days.

Re: Google Cloud Storage Nearline graduates to general availability

#44

Does anyone know why bandwidth costs for both AWS and Google have remained flat while storage has rapidly approached .01 cents a gb? Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?

From 2014-11: “Networking is a red alert situation for us right now,” explained Hamilton. “The cost of networking is escalating relative to the cost of all other equipment. It is Anti-Moore. All of our gear is going down in cost, and we are dropping prices, and networking is going the wrong way. That is a super-big problem, and I like to look out a few years, and I am seeing that the size of the networking problem is…

it i kind of topologically clear - with linear growth of the number of nodes, the fabric network architectures have to grow more than linear (and you have to have fabric if you'd like to have performance growing linear with number of nodes - otherwise there wouldn't be any sense in adding nodes after some point)

Re: Google Cloud Storage Nearline graduates to general availability

#45
post #41

Why mention an SLA if it's 99%? To me, "SLA" is an enterprise oriented term, which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind. 99% for a typical non-scaling enterprise app is crap.

> Why mention an SLA if it's 99%?

Because at least part of the target market for this product cares about quantifying the guarantee, whatever it is.

> To me, "SLA" is an enterprise oriented term,

I think that's excessively narrow in terms of who cares about it, but Nearline is clearly in large part an enterprise-targeted offering, so, even if it was a purely enterprise-oriented term, its appropriate.

> which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind.

Knowing expected failure characteristics can be an important input to intelligently architecting with failure in mind.

> 99% for a typical non-scaling enterprise app is crap.

Nearline isn't an app, its one of a set of closely related storage offerings that, by design, would usually be used in coordination with each other and possibly other storage systems by an app. For its role in that stack, 99% doesn't seem immediately unreasonable, to me.

Re: Google Cloud Storage Nearline graduates to general availability

#46

Earlier quoted context omitted.

But you are much more expensive than S3 & CloudFront when it comes to request pricing Cloud Storage considers retrieving an object through its HTTP url is considered a class B XML request type which is priced at $0.01/10,000 ops. This is 1.3x-2.5x more expensive than cloudfront and s3 respectively. This is also not well documented and caused me some trouble. I didn't expect HTTP GET requests to be counted as XML API…

What did you expect GET requests to be counted as then? As an aside, I think it's fairly well documented: https://cloud.google.com/storage/pricing#operations-pricing (lists "GET Object" under class B) [but disclosure: I work for Google]

It did not occur to me that HTTP GET will be considered as an XML API operation. So I didn't expect HTTP GET requests to be charged because everywhere I read (including that page you listed) the pricing is for XML API operation.

That + it being more expensive than cloudfront and s3 came as a rude shock :(

In general I find google cloud documentation and the service much better and more pleasant to work with than AWS but in this case it is not. Both S3 and Cloudfront have much clearer pricing (and positioning). In network pricing, both S3 & Cloudfront are significantly cheaper

Re: Google Cloud Storage Nearline graduates to general availability

#47
post #42

Earlier quoted context omitted.

To store 100PB, one can always buy 50.000 hard drives. Would only cost $2-3 million of initial payment. I wonder how much maintenance would cost to provide same availability with near storage.

Don't forget the full-time admins and hwops people. Operations costs are going to dwarf the capital investment.

Or you could leave it in the cloud where it already is and not transfer it to Google's cloud. I wasn't suggesting building your own...

Re: Google Cloud Storage Nearline graduates to general availability

#48

Earlier quoted context omitted.

What did you expect GET requests to be counted as then? As an aside, I think it's fairly well documented: https://cloud.google.com/storage/pricing#operations-pricing (lists "GET Object" under class B) [but disclosure: I work for Google]

It did not occur to me that HTTP GET will be considered as an XML API operation. So I didn't expect HTTP GET requests to be charged because everywhere I read (including that page you listed) the pricing is for XML API operation. That + it being more expensive than cloudfront and s3 came as a rude shock :( In general I find google cloud documentation and the service much better and more pleasant to work with than AWS…

Per my parent comment, comparing AWS networking functionality to Google's is Apples-to-Oranges.

Re: Google Cloud Storage Nearline graduates to general availability

#49
post #39
post #32

Earlier quoted context omitted.

(disclosure: am work for Google). Not sure on AWS. Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition: - Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler. - Google Compute Engine is a VPC…

"SSL added and removed here! :¬)"

http://arstechnica.com/information-technology/2013/11/google...

Re: Google Cloud Storage Nearline graduates to general availability

#50
post #15

6 months of free storage is a no brainer business strategy. Once you get anywhere near 100PB of data loaded, are you really going to move it elsewhere after 6 months?

We (rsync.net) maintain 's3cmd' and 'gsutil' in our environment, so you can do direct cloud cloud transfers from either amazon or google to and from rsync.net. We don't have a cold storage product, so perhaps not relevant for nearline ... but it is relevant for their online options. You do have the ability to migrate to another provider. Two random notes: - we're announcing zfs send/recv[1] as a transport in the next…

If you have tons of TBs, you don't want to care about random failure with some third party. Amazon S3 has the reputation that they never lost a file, AFAIK. On their scale that provides an argument to just fork over the cash and forget about it.

While I think ZFS is great, it's just not a solution for really large storage.

Also, if you are still using s3cmd, that only tells me how inexperienced you are.

Post reply on HN