Distributed Transactions at Scale in Amazon DynamoDB (2023)

Distributed Transactions at Scale in Amazon DynamoDB (2023)
muratbuffalo.blogspot.com

Sounds to me DynamoDB works well for well defined use-cases. That to me is a plus!

My items are not relations, and I don't see the point in transforming them to and from relational form. And if I did, each row would have like 5 columns set to NULL, in addition to a catch-all string 'data' column where I put the actual stuff I really need. Which is how you slow down an SQL database. So RDBMS is no good for me, and I'm no good for RDBMS.

RDBMS offers strong single-node consistency guarantees (which people leave off by default by using an isolation level of 'almost'!). But even without microservices, there are too many nodes: the DB, the backend, external partner integrations, the frontend, the customer's brain. You can't do if-this-then-that from the frontend, since 'this' will no longer be true when 'that' happens. So even if I happen to have a fully-ACID DB, I still lean into events & eventual consistency to manage state across the various nodes.

Given that I'm using more data than a naive CRUD/SQL app would (by storing events for state replication) and my data is stringy enough to kill my (and others') performance. So what's the solution? Make my read-writes completely independent from other read-writes - no joins, no foreign keys, etc.

The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it? For this reason I'd probably reach for Cassandra first. That said I haven't looked at the landscape in a while and there might be much better tools.

But it also wouldn't matter what I want to use instead of DynamoDB, because the DevOps team of wherever I work will just choose whatever's native&managed by their chosen cloud provider.

- In a SAAS API service we used dynamodb to look up API keys and track their daily usage data. It is fast enough to look up k/v pairs (api key => key info). And also aggregate small sets (We'd sum up call counts for current month and check if the API key had enough credits). This meant that the API itself did not need our RDBMS to function. We also had a postgresql instance for all relational data, subscriptions, user info etc. Had a trigger that would push any api key / subscription change to DynamoDB. In case of RDS issues, things kept chugging along.

- Working on a large buzzfeed like social media / news site in my country. We needed to store a lot of counters (reactions to articles, poll answers etc). All went into dynamodb and looked up from there. No hits on actual rdbms. There were a lot of traffic and dynamo made scaling things / keeping rds from melting easy for this kind of non critical data.

I'd not build an entire thing on DynamoDB but for specific use cases, I just loved it.