[deleted]
Google Cloud Spanner is now half the cost of Amazon DynamoDB
171–180 of 371 posts
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#172I'm torn because I have really liked Google offerings in the past (I'm pretty locked in on gmail, I have different things running on GCP already, etc). But I've also been feeling burned a bit by Google suddenly ending services. I had all my domains happily in Google domains until they recently sold it suddenly to Squarespace, who I'm not interested in dealing with. My phone is a Google Pixel and I was using the Googl…
> But I've also been feeling burned a bit by Google suddenly ending services.
The products/services that “Google” the search company launches are different than “Google Cloud”. While the discontinuation of Google products is annoying it has nothing to do with Google Cloud products/services. I don’t think Google Cloud abruptly announces discontinuing products/services as they have paid customers.
Regarding Google Domains that is a Google product. The equivalent product from Google is “Google Cloud Domains” which is available to Google Cloud customers.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#173“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
>This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generating this on DynamoDB, and not all of AWS is my read.
There's no indication that google is talking about ALL of spanner either? The examples they list are all internal google services, and they specifically say "inside google".
I'm also dubious that even with all of the AWS usage accounted for that DynamoDB tops Spanner if Amazon themselves are only at 126 million queries per second on Prime Day.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#174“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
From the AWS blog post they referenced- > DynamoDB powers multiple high-traffic Amazon properties and systems including Alexa, the Amazon.com sites, and all Amazon fulfillment centers. Over the course of Prime Day, these sources made trillions of calls to the DynamoDB API. DynamoDB maintained high availability while delivering single-digit millisecond responses and peaking at 126 million requests per second. Amazon w…
I used DynamoDB as part of the job a few years ago and never got single-millisecond responses - it was 20ms minimum and 70+ on a cold-start, but I can accept that optimising Dynamo's various indexes is a largely opaque process. We had to add on hacks like setting the request timeout to 5ms and keeping the cluster warm by submitting a no-op query every 500ms to keep it even remotely stable. We couldn't even use DAX because the Ruby client didn't support it. At the start we only had a couple of thousand rows in the table so it would have legit been faster to scan the entire table and do the rest in memory. Postgres did it in 5ms.
If Amazon said they didn't use DAX that day I would say they were lying.
The average consumer or startup is not going to squeeze out the performance of Dynamo that AWS is claiming that they have achieved.
In fact, it might have been fairer in Ruby if they didn't hard-code the net client (Net/HTTP). I imagine performance could have been boosted by injecting an alternative.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#175“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
Frankly it's a bit weird to see this kind of dick measuring in a product blog post from the "Director of Engineering" :/
s/the "Director of Engineering"/a "Director of Engineering"/
There are many engineering directors at Google.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#176And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else. AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Ju…
Except for projects for which NoSQL is a better fit than a RDBMS, no? If I'm writing a chat app with millions of messages and very little in the way of "relationships", should I use Postgres or some flavor of NoSQL? Honest question.
Either way can work. Getting to millions of messages is going to be the hard part, not storing them.
As with all data storage, the question is usually how do you want to access that data. I don't have experience with Postgres, but a lot of (older) experience with MySQL, and MySQL makes a pretty reasonable key-value storage engine, so I'd expect Postgres to do ok at that too.
I'm a big fan of pushing the messages to the clients, so the server is only holding messages in transit. Each client won't typically have millions of messages or even close, so you have freedom to store things how you want there, and the servers have more of a queue per user than a database --- but you can use a RDBMS as a queue if you want, especially if you have more important things to work on.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#177Does Google Cloud Spanner use the Spanner database per the 2013 paper? Or does Google simply use the brand and implement a cheaper and more performant db under the hood? I suspect it is the latter because most companies do not need global consistency anyway, so it may make sense to relax parts of what Spanner was originally built for.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#178Earlier quoted context omitted.
> I will say that it does show a vote of confidence to say that Photos, Gmail and Ads use GCP infra, I'm not sure? I guess I'm mostly not sure what "gcp infra" means there. The blog post says "Spanner is used ubiquitously inside of Google, supporting services such as; Ads, Gmail and Photos." But there's google-internal spanner, and gcp spanner. A service using spanner at Google isn't necessarily using gcp. (No clue a…
borg and k8s are completely unrelated bits of software with roughly similar goals. gcp spanner and normal spanner are different deployments of the same code.
>different deployments
Which can be the difference between 99.99% availability and 99% availability with data corruption issues. Not saying that's the case here but one should not downplay the difference deployments can make.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#179But a droplet with a Postgres db is nearly free, and seriously performant. And for 65/month, you can get a VERY beefy Hetzner server. You’ll have to wade through the crazy thicket that is the menu of cloud offerings. I gave that one look, and decided I might as well give up and learn the basics of Linux admin once and apply it for life.
> decided I might as well give up and learn the basics of Linux admin once and apply it for life.
Yeah I think this raises an underappreciated drawback of working on heavily AWS/GCP native projects. So much of the time ends up being spent on service level config and troubleshooting that has little relevance elsewhere.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#180And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else. AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Ju…
Except for projects for which NoSQL is a better fit than a RDBMS, no? If I'm writing a chat app with millions of messages and very little in the way of "relationships", should I use Postgres or some flavor of NoSQL? Honest question.
Postgres. NoSQL databases are specialized databases. They are best-in-class at some things, but generally that specialization came at great cost to their other options. DynamoDB is an amazing key-value store, but is severely limited at everything else. Elasticsearch is an amazing for search and analytics, but is severely limited at everything else. Other specialized databases that are SQL-full are also great at what they do, like Spark is a columnar database that has amazing capabilities for massive datasets where you need lots of cross-joins, but that severely limits it's ability to act in a lot of roles, because they traded latency for throughput and horizontal scalability, and you're restricted in what you can do with it.
The super-power of Postgres is that it supports everything. It's a best-in-class relational database, but it's also a decent key-value store, it's a decent full-text search engine, it's a decent vector database, it's a decent analytics engine. So if there's a chance you want to do something else, Postgres can act as a one-stop-shop and doesn't suck at anything but horizontal scaling. With partitioning improving, you can deal with that pretty well.
If you're writing fresh, there is basically no reason not to use Postgres to start with. It's only when you already know your scale won't work with Postgres that you should reach for a specialized database. And if you think you know because of published wisdom, I'd recommend you set up your own little benchmark, generate the volume of data you want to support, and then query it with Postgres and see if that is fast enough for you. It probably will be.