Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud Spanner is now half the cost of Amazon DynamoDB

cloud.google.com

Post reply on HN
Print view View original

261–270 of 371 posts

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#261

Post by Cthulhu_ »

dekhn wrote: post #143
gregdoesit wrote: post #82

Earlier quoted context omitted.

I tried to find it in this video, but failed. Could you please share a time stamp on where to look? It’s a pretty big deal if Gmail migrated to GCP-provided Spanner(not to an internal Spanner instance) and sounds like he kind of vote of confidence GCP and Cloud Spanner could benefit from: might I suggest to write about it? It’s easier to digest and harder to miss than an hour-long keynote video with no time stamps. A…

It's almost certainly not the case that Gmail uses Cloud Spanner rather than Internal Spanner. I don't think Cloud Spanner (or most of Google's cloud products) have the featureset required to support loads like Gmail (both in terms of technical capability, and security/privacy features). When I worked at Google I tried to get more services to migrate to the cloud but the internal environment that was built up over 25…

And yet, if they do, that's probably one of the best sales pitches they could have - dogfooding. After all, isn't that also how AWS started, just reselling the services and servers they already use themselves?

It doesn't make much sense to have a 'better' version of a product you sell but keep it internal.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#262

Post by insanitybit »

vicpara wrote: post #182

Just moved our infra from GCP to AWS. Kubernetes clusters, LB, storage, lambdas, KMS and all of it. Google runs their tech stack as if it's a startup that builds their CV. Everything is immature, tons of hacks, undocumented features. If you are on their k8s there are tons of upcoming new versions and features that force you to revisit key hacks you put in your infra because of their misgivings. Our infra team keeps t…

The horror stories of Google support, across all of their products, is enough for me to never trust GCP. Even if someone told me today "GCP is the exception, they have great support" I probably wouldn't care - they are so organizationally incapable of providing good support that, even if they did so today, I wouldn't believe that it could last.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#263

Post by tpmx »

rlpb wrote: post #245
packetslave wrote: post #75

Earlier quoted context omitted.

TWELVE YEARS AGO! find a new complaint.

In my world, most production services need to last longer than twelve years, so need relative pricing stability and no complete pricing restructures over a longer period. Perhaps your world is different.

I sort of agree, and I also posted other aspects, but please note this:

https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address...

We are introducing a new charge for public IPv4 addresses. Effective February 1, 2024 there will be a charge of $0.005 per IP per hour for all public IPv4 addresses, whether attached to a service or not

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#264

Post by agonz253 »

YetAnotherNick wrote: post #253
agonz253 wrote: post #227

Earlier quoted context omitted.

It's not purely a matter of cost, right? Say you want or need a highly available, high performance distributed database with externally consistent semantics. Are you going to handle the sharding of your Postgres data yourself? What replication system will you use for each shard? How will you ensure strong consistency? Will you be able to do transactions across shards? These are problems that systems like Spanner, Coc…

Just curious, why would distributed be design requirement? Is individual machine failure likely in AWS/GCP? The only failure I have seen in region level issues which spanner or dynamo don't help with AFAIK.

Individual machine failure is not likely, but we're hypothesizing the need for multiple shards for high performance. So now we have more machines and so the probability of failure increases. So we need to add replication, but then we need to deal with data getting out of sync, etc.... As others have mentioned though, these issues only really become important at a certain scale.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#265

Post by GabeWeiss_ »

nerpderp82 wrote: post #260
falcolas wrote: post #246

Earlier quoted context omitted.

We have an old "quiet part out loud" corporate story. It's about how one arm of Google using our service and wondering why it had so much downtime, only for us to point at their GAE arm and say "when they're down, we're down". They went and talked to GAE and - funny enough - were able to correlate the downtime they observed with GAE downtime. GAE uptime improved, for a little while. Yeah, we're on AWS now too.

Does Google run on GCP?

Parts, yes. In reference to the specifics mentioned in here though, those services run on Infra Spanner, not Cloud Spanner, but they're the same stack. The main reason things like Gmail, Ads, etc haven't swapped into GCP is because of the internal tooling that's built up around the infra spanner relating to those services specific to Google that don't make sense in Cloud Spanner.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#266

Post by lcw »

nerpderp82 wrote: post #260
falcolas wrote: post #246

Earlier quoted context omitted.

We have an old "quiet part out loud" corporate story. It's about how one arm of Google using our service and wondering why it had so much downtime, only for us to point at their GAE arm and say "when they're down, we're down". They went and talked to GAE and - funny enough - were able to correlate the downtime they observed with GAE downtime. GAE uptime improved, for a little while. Yeah, we're on AWS now too.

Does Google run on GCP?

From my understanding they don't dogfood a lot of gcp products internally. That's how you end up with janky integrations between their products. It's really frustrating at times to see their cloud architects pitch some grouping of technologies that you should use to find out the integrations aren't well tested at scale. For example, pushing for pubsub to be used with dataflow for near real time processing just to figure out at scale global pubsub has high latency, above 1 minute sometimes 5 minutes, on 1% of messages at scale.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#267

Post by jeffbee »

forrestbrazeal wrote: post #79
gregdoesit wrote: post #71

“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…

We shared some details about Gmail's migration to Spanner in this year's developer keynote at Google Cloud Next [0] - to my knowledge, the first time that story has been publicly talked about. [0] https://www.youtube.com/watch?v=268jdNwH6AM

Wow, almost content-free presentation! How obnoxious!

This wasn't the first time Gmail has replaced the storage backend in-flight. The last time, around 2011, they didn't hype it up, they called it "a storage software update" in public comms. And that other migration is the origin of the term "spannacle", because during that migration the accounts that resisted moving from [[redacted]] to [[redacted]] we called barnacles.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#268

Post by jeffbee »

Cthulhu_ wrote: post #261
dekhn wrote: post #143

Earlier quoted context omitted.

It's almost certainly not the case that Gmail uses Cloud Spanner rather than Internal Spanner. I don't think Cloud Spanner (or most of Google's cloud products) have the featureset required to support loads like Gmail (both in terms of technical capability, and security/privacy features). When I worked at Google I tried to get more services to migrate to the cloud but the internal environment that was built up over 25…

And yet, if they do, that's probably one of the best sales pitches they could have - dogfooding. After all, isn't that also how AWS started, just reselling the services and servers they already use themselves? It doesn't make much sense to have a 'better' version of a product you sell but keep it internal.

It makes sense because the public will not use the internal APIs which have non-standard wire protocols, weird authentication schemes, etc.

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#269

Post by iot_devs »

bennyg wrote: post #243
iot_devs wrote: post #233

Earlier quoted context omitted.

No need to guess when you can measure. I am running https://cloud-canary.com a service where I monitor AWS primary services for latency and availability. It comes with a lot of data. For instance this is the latency I see doing operations against Dynamo. https://cloudcanary.grafana.net/public-dashboards/c53e2092d6...

Nice dash - if you don't mind a drive-by recommendation: I use Grafana for work a lot and it's nice to see a table legend with min, max, mean, and last metrics for these kinds of dashboards. Really makes it easy to grok without hovering over data points and guessing.

Thanks a lot!

I'll definitely update it!

Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB

#270

Post by mianos »

RhodesianHunter wrote: post #217
jiggawatts wrote: post #158

Earlier quoted context omitted.

AWS will happily rent you a server with 24 TB of memory for about $200/hour. Columnar databases typically get a 10:1 compression ratio over raw data = 240 TB effectively. That’s a lot of data.

Which columnar databases are doing the above in-memory?

Kdb with ease, but down 500k on licenses and the cost of people who can use it. :)
Post reply on HN