Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed Transactions at Scale in Amazon DynamoDB (2023)

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

11–20 of 62 posts

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#11

Post by throwaway82452 »

mrkeen wrote: post #5
tbarbugli wrote: post #2

Using DynamoDB in 2025 is such a weird proposition. Horrible dev experience, no decent clients/libs, complex pricing, weird scaling in/out mechanism, slow, it only works well for well defined use-cases.

Very often I find myself wanting to store item(s) using a key. My items are not relations, and I don't see the point in transforming them to and from relational form. And if I did, each row would have like 5 columns set to NULL, in addition to a catch-all string 'data' column where I put the actual stuff I really need. Which is how you slow down an SQL database. So RDBMS is no good for me, and I'm no good for RDBMS.…

> The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it?

Amazon provides a downloadable version for development. I don't know how close it is to the real thing, but it makes it easier to do local dev.

Localstack also supports it in their paid version

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#12

Post by lambrospetrou »

llama-mini wrote: post #3
tbarbugli wrote: post #2

Using DynamoDB in 2025 is such a weird proposition. Horrible dev experience, no decent clients/libs, complex pricing, weird scaling in/out mechanism, slow, it only works well for well defined use-cases.

> it only works well for well defined use-cases. Sounds to me DynamoDB works well for well defined use-cases. That to me is a plus!

Exactly this.

DynamoDB is a pain in the ass if you want to do too many relational or arbitrary queries. It's not for data exploration.

It is my favourite database though (next to S3)! For cases where my queries are pretty much known upfront, and I want predictable great performance. As Marc Brooker wrote in [1], "DynamoDB’s Best Feature: Predictability".

I consistently get single digit millisecond GETs, 10-15ms PUTs, and a few more milliseconds for TransactWriteItems.

Are you able to complex joins? No. Are you able to do queries based on different hash/sort keys easily? Not without adding GSIs or a new table. The issue in the past few years was the whole craze around "single-table design". Folks took it literally as having to shove all their data in a single table, instead of understanding the reason and the cases that worked well. And with ongoing improvements of DynamoDB those cases were getting fewer and fewer over time.

But, that's what tradeoffs are about. With on-demand tables, one-shot transactions, actually serverless storage/scaling, and predictable performance you get very, very far.

1. https://brooker.co.za/blog/2022/01/19/predictability.html

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#13

Post by ndr »

narmiouh wrote: post #10
eknkc wrote: post #7

Earlier quoted context omitted.

2 times I have used DynamoDB and been extremely happy; - In a SAAS API service we used dynamodb to look up API keys and track their daily usage data. It is fast enough to look up k/v pairs (api key => key info). And also aggregate small sets (We'd sum up call counts for current month and check if the API key had enough credits). This meant that the API itself did not need our RDBMS to function. We also had a postgres…

Redis?

is it as easy to make that data durable?

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#15

Post by redwood »

qaq wrote: post #14

AWS needs to buy a proper newSQL databse Dynamo is just horribly outdated product.

What's your take on Aurora DSQL?

My take is anything single cloud provider proprietary and tabular in 2025 is going to over time feel too limited. Having a json column doesn't cut it. But I'm a believer in document databases

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#16

Post by eknkc »

narmiouh wrote: post #10
eknkc wrote: post #7

Earlier quoted context omitted.

2 times I have used DynamoDB and been extremely happy; - In a SAAS API service we used dynamodb to look up API keys and track their daily usage data. It is fast enough to look up k/v pairs (api key => key info). And also aggregate small sets (We'd sum up call counts for current month and check if the API key had enough credits). This meant that the API itself did not need our RDBMS to function. We also had a postgres…

Redis?

What would we gain from Redis in these use cases?

We used it extensively on the second project I mentioned and a couple of other projects for caching / rate limiting and distributed locking needs. Never enabled the persistence layer (which I believe is pretty durable). So we only treated as an ephemeral data store, lowering the architectural complexity of things significantly. Otherwise you need to think about backups, testing backups, clustering in case of scaling needs, I have no idea how persistence works with clustering... DynamoDB is fully managed and solid.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#17

Post by mejutoco »

eknkc wrote: post #16
narmiouh wrote: post #10

Earlier quoted context omitted.

Redis?

What would we gain from Redis in these use cases? We used it extensively on the second project I mentioned and a couple of other projects for caching / rate limiting and distributed locking needs. Never enabled the persistence layer (which I believe is pretty durable). So we only treated as an ephemeral data store, lowering the architectural complexity of things significantly. Otherwise you need to think about backup…

There are redis offerings that are fully managed as well. You have both options.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#18

Post by dygd »

throwaway82452 wrote: post #11
mrkeen wrote: post #5

Earlier quoted context omitted.

Very often I find myself wanting to store item(s) using a key. My items are not relations, and I don't see the point in transforming them to and from relational form. And if I did, each row would have like 5 columns set to NULL, in addition to a catch-all string 'data' column where I put the actual stuff I really need. Which is how you slow down an SQL database. So RDBMS is no good for me, and I'm no good for RDBMS.…

> The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it? Amazon provides a downloadable version for development. I don't know how close it is to the real thing, but it makes it easier to do local dev. Localstack also supports it in their paid version

The downloadable version is nowhere near ready for production. It's performance is also excruciatingly slow.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#19

Post by tempworkac »

dygd wrote: post #18
throwaway82452 wrote: post #11

Earlier quoted context omitted.

> The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it? Amazon provides a downloadable version for development. I don't know how close it is to the real thing, but it makes it easier to do local dev. Localstack also supports it in their paid version

The downloadable version is nowhere near ready for production. It's performance is also excruciatingly slow.

It doesn't really make any sense to use it locally - the whole point is that it's managed. If you just want a clustered key value store you could use Cassandra, Garnet, etc.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#20

Post by belter »

lambrospetrou wrote: post #12
llama-mini wrote: post #3

Earlier quoted context omitted.

> it only works well for well defined use-cases. Sounds to me DynamoDB works well for well defined use-cases. That to me is a plus!

Exactly this. DynamoDB is a pain in the ass if you want to do too many relational or arbitrary queries. It's not for data exploration. It is my favourite database though (next to S3)! For cases where my queries are pretty much known upfront, and I want predictable great performance. As Marc Brooker wrote in [1], "DynamoDB’s Best Feature: Predictability". I consistently get single digit millisecond GETs, 10-15ms PUTs,…

> DynamoDB is a pain in the ass if you want to do too many relational or arbitrary queries

You are using it wrong. And no, it's not irony.

Post reply on HN