Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Google Cloud Storage Nearline

googlecloudplatform.blogspot.com

Post reply on HN
Print view View original

111–120 of 188 posts

Re: Introducing Google Cloud Storage Nearline

#111

Post by vidarh »

gaadd33 wrote: post #72
vidarh wrote: post #13

Earlier quoted context omitted.

I don't know why you think so. AWS is not very price competitive. I make good money consulting on setup of more cost-effective alternatives to things like S3. The large cloud providers sell on brand recognition, convenience and trust, not cost. They may compete with each other on cost to try to cannibalise each others markets, but they're not even close to pushing the envelope on cost efficiency in terms of the price…

Just wondering, what is more cost effective than S3 with the same reliability? If you have your own servers I can see running zfs and replicating snapshots every few minutes to a remote machine but if you aren't, what else is competitive? I do agree that a lot of AWS services aren't cost effective though, especially at any sort of non trivial scale.

Setting up your own with any of Ceph, Gluster, Riak, Swift or similar depending on your specific needs. Heck, I've got setups where we've been served well for years with a combination of inotifywait, grep and rsync (combined to trigger rsync instantly on modification events). It really depends on your access patterns, and there are lots of potential savings from making use of domain specific knowledge for your specific system.

In general, you can beat AWS with 3x replication across multiple data centres even with renting managed servers, especially as your bandwidth use grows as AWS bandwidth prices are absolutely ridiculous (as in, anything from a factor of 5 to 20 above what you'll pay if you shop around and depending on your other requirements). Lease to own in a colo drops the price even further.

Anything from 1/3 to 1/2 of AWS costs is reasonable with relatively moderate bandwidth usage, with the cost differential generally increasing substantially the more you access the data.

Most people also don't have uniform storage needs. If you use your own setup, people tend to be able to cut substantially more in cost by reducing redundancy for data where it's not necessary etc.

See also the Backblaze article that mentioned Reed-Solomon - if your app is suitable for doing similar you can totally blow the S3 costs further out of the water that way.

Re: Introducing Google Cloud Storage Nearline

#112

Post by vidarh »

vgt wrote: post #78
gaadd33 wrote: post #72

Earlier quoted context omitted.

Just wondering, what is more cost effective than S3 with the same reliability? If you have your own servers I can see running zfs and replicating snapshots every few minutes to a remote machine but if you aren't, what else is competitive? I do agree that a lot of AWS services aren't cost effective though, especially at any sort of non trivial scale.

Google Cloud Storage is cheaper than Amazon S3: http://aws.amazon.com/s3/pricing/ https://cloud.google.com/storage/pricing

Google Cloud Storage is also massively overpriced.

Re: Introducing Google Cloud Storage Nearline

#113

Post by ghshephard »

sp332 wrote: post #17

Isn't the price a little high? $10/TB/mo is the same as Dropbox, and this has a lot fewer features than Dropbox.

Total apples/oranges. You are comparing a Consumer System which is focussed on providing folder synchronization to a system that is meant to be used at scale (and is scaleable).

Re: Introducing Google Cloud Storage Nearline

#114

Post by wongarsu »

gearhart wrote: post #9
blfr wrote: post #5

Earlier quoted context omitted.

There it is. Still, someone like 'cperciva could pool a lot of their customers' data together and offer high speeds even for small amounts of data.

http://xkcd.com/1494/ The unusual thing about this market is that by constantly improving the price point of their product but keeping their profit margins the same, AWS has forced every competitor in it to actually compete at the limits of their capability - so any scheme like this that got any serious traction would presumably be self-undermining. The economists will have considered all of the unused provided capac…

For a reference point: The hardware cost for storage servers with full redundancy is in the order of $0.10 per GB (plus electricity, rack space etc.). Buying your own hardware and throwing it away every year is price competitive with Nearline (and you get full online storage, no three second delays and slow retrieval).

Of course there's good reasons not to roll your own solution until you reach a certain scale, but it's certainly possible to compete with Google and AWS on price.

Re: Introducing Google Cloud Storage Nearline

#115

Post by ghshephard »

manacit wrote: post #31
sp332 wrote: post #21

Earlier quoted context omitted.

Exactly, so why would the lower-level service aimed at big slow transfers be more expensive than a full-featured product?

There's a few reasons: * Dropbox doesn't guarantee 11 9s like this service does - when I'm backing critical data up, I want to make sure it's _there_. * Dropbox likely wouldn't take kindly to me storing 10PB, whereas that is what this service is designed for. * I've got SLAs and guaranteed speeds with this, Dropbox isn't designed for me to suddenly download 10PB very quickly.

Do you have any information to suggest that the Google Nearline service can handle 10PB?

Re: Introducing Google Cloud Storage Nearline

#116

Post by AceJohnny2 »

panarky wrote: post #50
fuzzythinker wrote: post #35

Earlier quoted context omitted.

Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo.

Glacier pricing is surprisingly complicated, and the actual cost can be much higher than $0.01 per GB-month if you don't read the fine print. The biggest gotcha is that you can only access 0.17% of the data you've stored in any given day without extra charges. So if you've stored 1000 GB, you can only access 1.7 GB per day for free. The cost for going over your daily "retrieval allowance" can be large, because cost i…

Thanks for the detailed breakdown, that's really interesting! Sounds like you were bitten by this? :)

Re: Introducing Google Cloud Storage Nearline

#117

Post by StavrosK »

sreitshamer wrote: post #68
moe wrote: post #36

This is a nail in the coffin of the SaaS backup industry (Backblaze, RescuePlan, etc.). With storage becoming essentially free the last excuse for not using self-hosted, secure backup tools like Arq[1] disappears. [1] http://www.haystacksoftware.com/arq/

I'm adding support for creating nearline buckets in Arq right now.

This is exactly what I want for Linux (headless), but I haven't been able to find anything like it. The closest thing I've found is attic, which, while amazing, requires a daemon to be running on the remote end (which is what makes checking, deduplication and deltas fast).

Does anyone know an Arq alternative for linux?

Re: Introducing Google Cloud Storage Nearline

#118

Post by VikingCoder »

pfortuny wrote: post #104
libria wrote: post #81

Earlier quoted context omitted.

If we place too many restrictions on how companies should offer preview releases, they'll just stop entirely. Are you suggesting that even those comfortable taking on risk to get a sneak peak should be forced to wait for general availability?

No, one simply should wait for companies to offer services with guarantees for one's own data, if the data is important. Just that.

If they lose your data, yes, that's one thing.

But if the 3 seconds becomes 6 seconds... Or the price goes up... Or they announce they're end-of-life'ing the product...

Just move your data somewhere else.

Sure, it'd be inconvenient (and maybe expensive) to move. So, you balance all of that out in your mind, and maybe this is the right service for you, and maybe it's not.

Re: Introducing Google Cloud Storage Nearline

#119

Post by vgt »

ghshephard wrote: post #115
manacit wrote: post #31

Earlier quoted context omitted.

There's a few reasons: * Dropbox doesn't guarantee 11 9s like this service does - when I'm backing critical data up, I want to make sure it's _there_. * Dropbox likely wouldn't take kindly to me storing 10PB, whereas that is what this service is designed for. * I've got SLAs and guaranteed speeds with this, Dropbox isn't designed for me to suddenly download 10PB very quickly.

Do you have any information to suggest that the Google Nearline service can handle 10PB?

If you'd like to put 10PB in storage, let's talk offline :)

source: SE at Google Cloud

Re: Introducing Google Cloud Storage Nearline

#120

Post by vgt »

fivedogit wrote: post #96

Some of these Google/Amazon/Apple "Introducing Product X..." stories are starting to strike me as intellectually devoid, creeping into "free advertising" territory. This particular story is about a Google service that is an extremely late entrant into the market (Amazon Glacier, etc) and offers almost nothing new. I'm not saying it's a bad product or not useful to folks, but it's not, by any means, groundbreaking. I'…

I suggest you read, for example, this article from TC:

http://techcrunch.com/2015/03/11/google-launches-cloud-stora...

This information should challenge your argument that Nearline is not groundbreaking, at least in the "cloud archival storage" space.

Post reply on HN