Live data from Hacker News

Google Cloud Storage Nearline graduates to general availability

googlecloudplatform.blogspot.com

21–30 of 58 posts

Re: Google Cloud Storage Nearline graduates to general availability

#21
post #3

It looks like an amount of space hard to fill with meaningful data. I wonder if they would prevent people from filling it with random junk just for fun.

Only hard to fill if you're dealing with text only. Say you're building a video copyright detection crawler and you want to store videos from all over the internet.

An HD video stream from Netflix uses about 3GB per hour. So 100PB gives you about 4000 years of content.

Re: Google Cloud Storage Nearline graduates to general availability

#22
post #4

It's basically impossible to upload this data in 6 months. 100 PB = 102400 TB = 104857600 GB = 107374182400 MB. You have 365/2 * 24 * 3600 seconds in 6 months, that's 15768000 seconds. So you need to upload ~7 GB/sec, that's 60 Gb/sec constant upload.

Google also released today "Google Storage Transfer Service", which can help you reach this scale.

Re: Google Cloud Storage Nearline graduates to general availability

#23

As cute as it is to have $.01/GB backup storage....every time you pull out a backup it costs you $.13/GB. That is a pretty steep hit when you can use a lower quality object store: https://www.runabove.com/storage/object-storage.xml For $.01/GB and $.01/GB to pull. Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 1…

Pulling out backup on the site is $.01/GB - am I missing something?

Re: Google Cloud Storage Nearline graduates to general availability

#24
post #14

I am absolutely blown away with the 100 PB for every customer... Meaning all together they must have somewhere in the exabyte to zettabyte range or are completely betting on the fact that no one will be able to/actually be willing fill this/their cloud. I guess I am just not used to the fact that on average most people have at least a 1 TB hard drive or hard drives that add up to 1 TB...

> I am absolutely blown away with the 100 PB for every customer...

Its 100PB free for 1 month, if you commit to transfer in at least 1PB in the first three months, and commit to keep at least 1PB for 12 months (Its "up to 6 months" free of 100PB, but more than 1 month free requires greater than 1PB commitment.) At $0.01/GB/mo, 1PB of data for 11 months (12 months minus 1 month free) is a $110k commitment.

Its not "100PB for every customer", at least not free, its a variable size discount that covers up to 100PB for 1-6 months of a 12 month commitment period for customers able to make up-front commitments of $110k or more.

Re: Google Cloud Storage Nearline graduates to general availability

#25
post #14

I am absolutely blown away with the 100 PB for every customer... Meaning all together they must have somewhere in the exabyte to zettabyte range or are completely betting on the fact that no one will be able to/actually be willing fill this/their cloud. I guess I am just not used to the fact that on average most people have at least a 1 TB hard drive or hard drives that add up to 1 TB...

1. It's only free for "up to" 6 months, and a client that moved 100 PB is not wanting to migrate away after 6 months merely to play on the free temporary space (at $0.01 per GB per month, that's what a million bucks a month starting on the 7th month ? That's a great incentive to give it free as a starter)

2. It's only if you upload at least 1 PB of data in the first three month, which removes a lot of companies and only leaves the serious heavy data potential customers

Re: Google Cloud Storage Nearline graduates to general availability

#26
post #23

As cute as it is to have $.01/GB backup storage....every time you pull out a backup it costs you $.13/GB. That is a pretty steep hit when you can use a lower quality object store: https://www.runabove.com/storage/object-storage.xml For $.01/GB and $.01/GB to pull. Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 1…

Pulling out backup on the site is $.01/GB - am I missing something?

If I am reading the pricing page[1] correctly, you still get hit with the egress charges on top of the $0.01/GB charge to retrieve from Nearline. Thus, for less than 1TB of data retrieval is $0.13/GB ($0.01 Nearline charge + $0.12 egress charge) if you are not in China or Australia.

[1] https://cloud.google.com/storage/pricing#storage-pricing

Re: Google Cloud Storage Nearline graduates to general availability

#27
post #16

Earlier quoted context omitted.

How much space a site like imgur needs?

Its a different use case, cold storage vs content delivery...

Not really.

Cold storage is absolutely a necessity when you reach the scale of imgur, unless you want to run the service incredibly inefficiently.

imgur keeps photos forever [0]. To do so without some kind of cold storage infrastructure in place for all the millions of photos that will probably never be seen again, or perhaps at a rate of once per year, would be infrastructure cost suicide.

Facebook faces a similar challenge and has gone in to detail about how it developed a cold storage strategy to deal with it[1].

I would say that for a service like imgur there is probably a level of parity between the amount of importance they need to place on content delivery and on cold storage.

[0] https://help.imgur.com/hc/en-us/articles/201476457-How-long-...

[1] https://code.facebook.com/posts/1433093613662262/-under-the-...

Re: Google Cloud Storage Nearline graduates to general availability

#28

As cute as it is to have $.01/GB backup storage....every time you pull out a backup it costs you $.13/GB. That is a pretty steep hit when you can use a lower quality object store: https://www.runabove.com/storage/object-storage.xml For $.01/GB and $.01/GB to pull. Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 1…

However, the egress charge only applies to data moved off Google Cloud Services. If you use a Google compute instance to verify the backup (same size as the Linode at $0.05/hr), the cost becomes:

(7 * 50 * .01) + (30 * .05) + (30 * .01 * 50) = $20.00/month

Re: Google Cloud Storage Nearline graduates to general availability

#30
post #19

From the details (in an image, some in fine print on the image) on the offer: that's if you migrate at least 1PB into Google Cloud Nearline in the first 3 months, and maintain at least 1PB in Google Clound Nearline for at least 12 months. And its only 1 month free at that level of commitment; anything more than 1 month requires a migration commitment to more than the 1PB. EDIT: At the time this comment was written, t…

Once the free period is over the 100PB will cost $1 million per month in storage fees alone. Will the free month(s) even cover the retrieval and transfer fees to take 100PB out of another cloud?

To store 100PB, one can always buy 50.000 hard drives. Would only cost $2-3 million of initial payment. I wonder how much maintenance would cost to provide same availability with near storage.
Post reply on HN