Earlier quoted context omitted.
I know the spanner marketing blurb says you can scale down etc. But I think in practice spanner is primarily aimed at use cases where you'd struggle to fit everything in a single postgres instance. Having said that I guess I broadly agree with your comment. It seems like a lot of people like to plan for massive scale while they have a handful of actual users.
I said this in another comment, but I have seen _two_ applications in my career that actually had a request load that might warrant something like one of these databases. One was an application with double digit million MAU and thousands of RPS on a very shardable data set, which fit Spanner's ideal access pattern and performance profile pretty well, but we paid an absolute arm and a leg for the privilege and ended u…
Google Cloud Spanner is now half the cost of Amazon DynamoDB
231–240 of 371 posts
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#23299% of the projects dont need dynamodb/spanner/ScyllaDB
• Spanner is more in the domain of CockroachDB — distributed strong consistency and ACID compliance. Both are ANSI SQL.
• DynamoDB and ScyllaDB are in the NoSQL key-value store / wide column domain. ScyllaDB is API compatible with DynamoDB, but is also API compatible with Cassandra Query Language (CQL). These databases are for eventual consistency use cases.
As to the "99%" comment, there are over 415 databases currently tracked on DB-engines.com (https://db-engines.com/en/ranking). It is a very competitive and specialized industry. Very few databases have more than 1% marketshare these days.
According to this: https://6sense.com/tech/relational-databases
...only the top 9 options show >1% of marketshare.
MySQL being so far in the lead (42%) is far more likely due to it being baked into so many OEM deals (over 2,000 as per https://www.mysql.com/oem/). Like, every WordPress site in the world than anything else.
Yet that doesn't really make MySQL great for every use cases. It has obvious limitations.
We also have to note that different studies have far different results.
In this video: https://statisticsanddata.org/data/the-most-popular-database...
It shows a different set of top databases, and has 11 of them showing >2% of marketshare. It has MySQL at only 13.91% of marketshare, with Oracle being on top at 31.19%.
The source of this poll is TOPDB Top Database index:
https://pypl.github.io/DB.html
It's not based on money marketshare, nor on poll results, but simply is counting how often a database is searched on Google. Note that anything that could garner 1% of the market would suddenly catapult that database into the top 16th place on the list.
At the end of the day, database "market share" — however it is calculated — is not representative of exclusive percentages. Many companies run more than 1 type of database. There might even be multiple SQL and NoSQL databases, each designed for purpose, at large enterprises. OLTP, OLAP. I would not be surprisedi in the least to learn that large corporations had well over 100 different databases running at any given time.
So, back to your basic comment: if a database is useful for even 1% of the use cases, that would still place it in the top 20 databases in the industry.
Moreover, a lowest-common denominator database will not be able to support many critical use cases where you really do need a specialized data model, or index type, or query language, or workload type, or latency, or distribution, or scale of QPS/TPS/OPS, or total data size or query payload size, etc.
I have likened the database industry to the state of Christianity after the Reformation. Even if there are still plenty of "Catholics" (classic SQL adherent, like Oracle), there are also Reformed faiths ("NewSQL" / "Distributed SQL", e.g., PostgreSQL), plus any number of Protestant reformation alternatives (NoSQL). I'm not sure what the "Orthodox" in this analogy refers to. Maybe SQL data warehouses? The schism between OLTP and OLAP? Dunno.
Anyway, it's an analogy. Bound to break down at some point.
At the end of the day, my advice: use the database most closely aligned to the dataset and workload and use case you have at hand. Don't just throw something at a problem because it's a popular choice, or because it's the database you happen to be most familiar with.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#233Earlier quoted context omitted.
From the AWS blog post they referenced- > DynamoDB powers multiple high-traffic Amazon properties and systems including Alexa, the Amazon.com sites, and all Amazon fulfillment centers. Over the course of Prime Day, these sources made trillions of calls to the DynamoDB API. DynamoDB maintained high availability while delivering single-digit millisecond responses and peaking at 126 million requests per second. Amazon w…
I used DynamoDB as part of the job a few years ago and never got single-millisecond responses - it was 20ms minimum and 70+ on a cold-start, but I can accept that optimising Dynamo's various indexes is a largely opaque process. We had to add on hacks like setting the request timeout to 5ms and keeping the cluster warm by submitting a no-op query every 500ms to keep it even remotely stable. We couldn't even use DAX be…
I am running https://cloud-canary.com a service where I monitor AWS primary services for latency and availability.
It comes with a lot of data.
For instance this is the latency I see doing operations against Dynamo.
https://cloudcanary.grafana.net/public-dashboards/c53e2092d6...
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#234Google also has a history of massively spiking the cost of its services. Vendor lockin is a dangerous thing.
Curious: have they ever increased prices of existing Google Cloud services? As far as I know, AWS only ever decreases prices on services for example
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#235“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
> At Amazon, practically every service is built on top of AWS is that true finally? It sure wasn't in the 2020-2021 timeframe.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#236Just moved our infra from GCP to AWS. Kubernetes clusters, LB, storage, lambdas, KMS and all of it. Google runs their tech stack as if it's a startup that builds their CV. Everything is immature, tons of hacks, undocumented features. If you are on their k8s there are tons of upcoming new versions and features that force you to revisit key hacks you put in your infra because of their misgivings. Our infra team keeps t…
Which is on brand with Google. They have no problem launching stuff, and no problem killing stuff. But man, then just get out of the cloud business and focus on what you're good at.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#237Earlier quoted context omitted.
> AWS support is so nice, you can't believe it. This reminds me of the fond days of having weekly customers calls. We develop AWS services, and we answer our customer-support calls directly. No middle man. Just techies to techies. And we made promises to customers on the fly, and customers sometimes project managed us.
Sounds like hell...
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#238“ According to the Amazon Prime Day blog post, DynamoDB processes 126 million queries per second at peak. Spanner on the other hand processes 3 billion queries per second at peak, which is more than 20x higher, and has more than 12 exabytes of data under management.” This comparison seems to be not exactly fair? Amazon’s 126 million queries per second was purely for Amazon-related services serving Prime Day generatin…
> At Amazon, practically every service is built on top of AWS - a proper vote of confidence! Not only this, but practically most, if not all, of the AWS services use DynamoDB, including use cases that are usually not for databases, such as multi-tenant job queues (just search "Database as a Queue" to get the sentiment). In fact, it is really really hard to use any relational DB in AWS. I mean, a team would have to go…
Edit: It's possible you're limiting your statement specifically to AWS teams, which would make it more accurate, but I read the use of "Amazon" in the quote you were replying to as including things like retail as well, etc.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#239Just moved our infra from GCP to AWS. Kubernetes clusters, LB, storage, lambdas, KMS and all of it. Google runs their tech stack as if it's a startup that builds their CV. Everything is immature, tons of hacks, undocumented features. If you are on their k8s there are tons of upcoming new versions and features that force you to revisit key hacks you put in your infra because of their misgivings. Our infra team keeps t…
I wonder what makes us different, I work in europe on video games; AWS’s handling of me when I was at Ubisoft left a really sour taste - when I moved into Tencent/Sharkmob I tried really hard to love AWS as it was the defacto industry standard and instead I was left with a feeling that most of it is inconsistent garbage papered over with lambda functions. I referred to these weird gotchas as “3am topics”; things that I don't have the mental capacity to deal with at 3am and convinced the studio to switch to GCP- which, incidentally they are still extremely grateful to me for doing.
Re: Google Cloud Spanner is now half the cost of Amazon DynamoDB
#240Just moved our infra from GCP to AWS. Kubernetes clusters, LB, storage, lambdas, KMS and all of it. Google runs their tech stack as if it's a startup that builds their CV. Everything is immature, tons of hacks, undocumented features. If you are on their k8s there are tons of upcoming new versions and features that force you to revisit key hacks you put in your infra because of their misgivings. Our infra team keeps t…