Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed Transactions at Scale in Amazon DynamoDB (2023)

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#41

Post by mlhpdx »

GiorgioG wrote: post #39

DynamoDB is great if you want to fork over a lot of money to AWS on a monthly basis. It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. There are so many better, less expensive options out there for 99.99999% of use-cases. Friends don't let friends use DynamoDB.

Not my experience at all. The DX is different, but not bad in my, and it has been extremely reliable and inexpensive for my use cases and workloads. I built mainstream stuff and a global multi-region replicated DDB table with backups is essentially free.

That said, I don’t do query time aggregation anymore, which seems to be a common challenge folks hit. Other approaches that implement streaming/incremental aggregation and make it look roughly like query time probably are simpler.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#42

Post by belter »

GiorgioG wrote: post #39

DynamoDB is great if you want to fork over a lot of money to AWS on a monthly basis. It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. There are so many better, less expensive options out there for 99.99999% of use-cases. Friends don't let friends use DynamoDB.

> It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling.

This attitude of everybody is incompetent at ...I need to teach them, is what is mind-boggling.

Your statement misunderstands the design principles of a system like this one or others similar. It's precisely those design principles that led to not allowing to run arbitrary queries on the backend.

You are trying to optimize for the 99.9th percentiles at massive scale...

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#43

Post by GiorgioG »

belter wrote: post #42
GiorgioG wrote: post #39

DynamoDB is great if you want to fork over a lot of money to AWS on a monthly basis. It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. There are so many better, less expensive options out there for 99.99999% of use-cases. Friends don't let friends use DynamoDB.

> It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. This attitude of everybody is incompetent at ...I need to teach them, is what is mind-boggling. Your statement misunderstands the design principles of a system like this one or others similar. It's precisely those design principles that led to not allowing to r…

No, it's the morons who pick a technology (in this case DynamoDB) without understanding it is the wrong tool for the job at hand, leaving the rest of us holding the bag for years on end for their poor choice of tools.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#44

Post by mike_hearn »

mrkeen wrote: post #5
tbarbugli wrote: post #2

Using DynamoDB in 2025 is such a weird proposition. Horrible dev experience, no decent clients/libs, complex pricing, weird scaling in/out mechanism, slow, it only works well for well defined use-cases.

Very often I find myself wanting to store item(s) using a key. My items are not relations, and I don't see the point in transforming them to and from relational form. And if I did, each row would have like 5 columns set to NULL, in addition to a catch-all string 'data' column where I put the actual stuff I really need. Which is how you slow down an SQL database. So RDBMS is no good for me, and I'm no good for RDBMS.…

Some RDBMS only offer single node consistency but others can scale write masters horizontally (e.g. Oracle).

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#45

Post by plandis »

tempworkac wrote: post #19
dygd wrote: post #18

Earlier quoted context omitted.

The downloadable version is nowhere near ready for production. It's performance is also excruciatingly slow.

It doesn't really make any sense to use it locally - the whole point is that it's managed. If you just want a clustered key value store you could use Cassandra, Garnet, etc.

I think people mostly use it for unit testing functionality since it’s generally a faster dev loop compared to running integration tests.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#46

Post by qaq »

belter wrote: post #21
qaq wrote: post #14

AWS needs to buy a proper newSQL databse Dynamo is just horribly outdated product.

> AWS needs to buy a proper newSQL databse Dynamo is just horribly outdated product. I am sorry but engaging in good faith, can you quality a little bit? Are you aware DynamoDB works as tier one product within AWS? Meaning it's one of the core pillars of the implementation of many other products? Did you look at these references?: https://aws.amazon.com/dynamodb/customers/ Have you seen the real world use?: https://a…

very low limit on number of keys in transaction very low limit on ops/sec on a single key extremely cumbersome to change the data layout development experience is just abysmal

Before Spanner, Cockroach etc. for some workloads you didn't have alternatives but that time is long gone.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#47

Post by qaq »

redwood wrote: post #15
qaq wrote: post #14

AWS needs to buy a proper newSQL databse Dynamo is just horribly outdated product.

What's your take on Aurora DSQL? My take is anything single cloud provider proprietary and tabular in 2025 is going to over time feel too limited. Having a json column doesn't cut it. But I'm a believer in document databases

Honestly haven't really had to deal with use case where json column does not cut it, but I would imaging there might be some.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#48

Post by pdhborges »

davidjfelix wrote: post #34
smashedtoatoms wrote: post #30

I came here for the bad takes, and I have not been disappointed. Dynamo slays when you know your access patterns and need consistent performance and no operations requirements. Turns out, that's the case most of the time. Think about it as application state instead of a db. It's not key-value like Redis. GSIs with compound keys allow access to data across multiple dimensions on virtually unlimited data with consisten…

Agreed. It's wild to me how many people think they need arbitrary queries on their transactional database and then go write a CRUD app with no transactional consistency between resources and everything is a projection from a user or org resource -- you can easily model that with Dynamo. You can offload arbitrary analytical queries or searches to a different database and stop conflating that need with your app's core…

Well my experience has always been the opposite. New query patterns are always appearing. The difference between an OLTP and an OLAP query is not as clear cut as one might imagine that justifies huge changes to an existing system.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#49

Post by mulmen »

mrkeen wrote: post #36
Lapapapaja wrote: post #28

Earlier quoted context omitted.

> I still lean into events & eventual consistency to manage state across the various nodes. You can get really far with a RDMS before event sourcing etc is needed, the benefit being both your dev and user experience are going to be much simpler and easier. If you already know your problem domain and scaling concerns up front sure. But starting with a scalable pattern like this is a premature optimization otherwise an…

> You can get really far with a RDMS before event sourcing etc is needed You can manage up to 0 partners easily. Once you go above that threshold, you're into "2-Generals" territory. At that point you're either inconsistent, eventually-consistent, or you're just bypassing your own database and using theirs directly. > dev and user experience are going to be much simpler and easier. I have objects, not relations. I'm…

Yeah this is a much better initial dev experience but you still have a schema, even if you ignore it. When your objects are inconsistently shaped something has to fix them. That something is going to take the shape of custom code that would make even the Perl-iest DBA blush.

Re: Distributed Transactions at Scale in Amazon DynamoDB (2023)

#50

Post by UltraSane »

rednafi wrote: post #31

No, thank you. DynamoDB promotes a terrible dev workflow, and there’s no way around it. It’s like embedding S3 directly into your application code. The client libraries are gigantic, and the documentation is misleading at times. Plus, Dynamo expects your access patterns to be static, which isn’t true most of the time. Hyperscaling is great, but many aren’t willing to give up everything else just for that.

I tried using it once and designing a schema for it makes relational databases seem fun by comparison.
Post reply on HN