"Organizations of all sizes and across all industries are increasingly looking to accelerate digital transformation and power AI-driven innovation."
How did Google become like this?
Google Cloud Spanner is now half the cost of Amazon DynamoDB
141–150 of 371 posts
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#142DynamoDB is such a good name though; "Cloud Spanner" doesn't tell me anything about what this does...
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#143Earlier quoted context omitted.
We shared some details about Gmail's migration to Spanner in this year's developer keynote at Google Cloud Next [0] - to my knowledge, the first time that story has been publicly talked about. [0] https://www.youtube.com/watch?v=268jdNwH6AM
I tried to find it in this video, but failed. Could you please share a time stamp on where to look? It’s a pretty big deal if Gmail migrated to GCP-provided Spanner(not to an internal Spanner instance) and sounds like he kind of vote of confidence GCP and Cloud Spanner could benefit from: might I suggest to write about it? It’s easier to digest and harder to miss than an hour-long keynote video with no time stamps. A…
It's almost certainly not the case that Gmail uses Cloud Spanner rather than Internal Spanner. I don't think Cloud Spanner (or most of Google's cloud products) have the featureset required to support loads like Gmail (both in terms of technical capability, and security/privacy features).
When I worked at Google I tried to get more services to migrate to the cloud but the internal environment that was built up over 25 years is much better at supporting billion+ users with private data.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#144Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#145Earlier quoted context omitted.
Postgres and Spanner do different things, in different ways, with different costs, risks, and implications. You could "just use" anything that is completely different and slightly cheaper. You could use a GitHub repository and just store your records as commits, for free, that's plenty cheap and works for small projects. But not really the same thing, is it?
My point is that I've seen very, very few situations (I can think of two in my entire career so far) where a "hyperscale NoSQL database" was actually the right choice to solve the problem. I find that a lot of folks turn to these databases for imagined scale needs, not actual hard problems that need solving.
Spanner has a SQL interface, fyi.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#146“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
> What would have perhaps been a more fair comparison is to share the peak load that Google services running on GCP generated on Spanner, and not the sum of their cloud platform. Not necessarily about volume of transactions, but this is similar to one of my pet-peeves with statements that use aggregated numbers of compute power. "Our system has great performance, dealing 5 billion requests per second" means nothing i…
Put yourself in the shoes of who they're targeting with that.
Probably dealing with thousands of requests per seconds, but wants to say they're building something that can scale to billions of requests per second to justify their choices, so there they go.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#147And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else. AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Ju…
I know the spanner marketing blurb says you can scale down etc. But I think in practice spanner is primarily aimed at use cases where you'd struggle to fit everything in a single postgres instance. Having said that I guess I broadly agree with your comment. It seems like a lot of people like to plan for massive scale while they have a handful of actual users.
I said this in another comment, but I have seen _two_ applications in my career that actually had a request load that might warrant something like one of these databases. One was an application with double digit million MAU and thousands of RPS on a very shardable data set, which fit Spanner's ideal access pattern and performance profile pretty well, but we paid an absolute arm and a leg for the privilege and ended up implementing a distributed cache in front of Spanner to reduce costs. The other just kept the data set in memory and flushed to disk/S3 backup periodically because in that case liveness was more important than completeness.
In the first case, the database created as many problems as it solved (which is true of any large application running at scale; your data store will _always_ be suboptimal). A fancy, expensive NoSQL database won't save you from solving hard engineering problems. At smaller scales (on the order of tens-hundreds of RPS), it's hard to go wrong with any established SQL (or open source NoSQL if that floats your boat) database, and IMO Postgres is the most stable and best bang for your engineering buck feature wise.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#148And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else. AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Ju…
Except for projects for which NoSQL is a better fit than a RDBMS, no? If I'm writing a chat app with millions of messages and very little in the way of "relationships", should I use Postgres or some flavor of NoSQL? Honest question.
Golden Rule of data: Use PostgreSQL unless you have an extremely good reason not to.
PostgreSQL is extremely good at append-mostly data, i.e like a chat log and has powerful partitioning features that allow you to keep said chat logs for quite some time (with some caveats) while keeping queries fast.
Generally speaking though PostgreSQL has powerful features for pretty much every workload, hence the Golden Rule.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#149And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else. AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Ju…
just migrated off of PG to ddb as the main db for my application (still copying data to SQL for data analytics). Working with distributed functions and code hosted on lambdas, the connection management to SQL became a nightmare with dropped requests all over the place.
Sacrificing a real DB to use serverless is exactly why serverless makes zero sense.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#150Earlier quoted context omitted.
My point is that I've seen very, very few situations (I can think of two in my entire career so far) where a "hyperscale NoSQL database" was actually the right choice to solve the problem. I find that a lot of folks turn to these databases for imagined scale needs, not actual hard problems that need solving.
Spanner has a SQL interface, fyi.
I don't think the API you interface with fundamentally changes the point that Spanner is hard to recommend from an engineering perspective at anything except the absolute most massive of scales, and even then it will create nearly as many problems as it solves. I'm not saying spanner is _wrong_ or shouldn't exist, but it's very difficult to be in the position where Spanner is the critical key to your application's success and not replaceable by .