Distributed Transactions at Scale in Amazon DynamoDB (2023)

Post by unethical_ban » Thu, Jan 16, 2025, 3:57 AM UTC

belter wrote: post #42

GiorgioG wrote: post #39

DynamoDB is great if you want to fork over a lot of money to AWS on a monthly basis. It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. There are so many better, less expensive options out there for 99.99999% of use-cases. Friends don't let friends use DynamoDB.

> It's inability to allow you to query on arbitrary attributes/fields without creating a separate index (i.e. copy of the entire collection) is mind-boggling. This attitude of everybody is incompetent at ...I need to teach them, is what is mind-boggling. Your statement misunderstands the design principles of a system like this one or others similar. It's precisely those design principles that led to not allowing to r…

Post by eknkc » Thu, Jan 16, 2025, 8:43 AM UTC

rad_gruchalski wrote: post #32

eknkc wrote: post #7

Earlier quoted context omitted.

2 times I have used DynamoDB and been extremely happy; - In a SAAS API service we used dynamodb to look up API keys and track their daily usage data. It is fast enough to look up k/v pairs (api key => key info). And also aggregate small sets (We'd sum up call counts for current month and check if the API key had enough credits). This meant that the API itself did not need our RDBMS to function. We also had a postgres…

> We also had a postgresql instance for all relational data, subscriptions, user info etc. Had a trigger that would push any api key / subscription change to DynamoDB. Wouldn't doing it right there in postgres limit your footprint?

Post by Kinrany » Thu, Jan 16, 2025, 9:48 AM UTC

rednafi wrote: post #52

UltraSane wrote: post #50

Earlier quoted context omitted.

I tried using it once and designing a schema for it makes relational databases seem fun by comparison.

Relational databases are indeed fun. There's no such thing as schemaless data; the schema either lives in your database or in your code. In the case of Dynamo, the schema lives in your code, determined by your access patterns. The problem is that access patterns change, and this state of extreme denormalization is difficult to adapt to evolving business requirements.

Post by mrkeen » Thu, Jan 16, 2025, 10:22 AM UTC

mulmen wrote: post #49

mrkeen wrote: post #36

Earlier quoted context omitted.

> You can get really far with a RDMS before event sourcing etc is needed You can manage up to 0 partners easily. Once you go above that threshold, you're into "2-Generals" territory. At that point you're either inconsistent, eventually-consistent, or you're just bypassing your own database and using theirs directly. > dev and user experience are going to be much simpler and easier. I have objects, not relations. I'm…

Yeah this is a much better initial dev experience but you still have a schema, even if you ignore it. When your objects are inconsistently shaped something has to fix them. That something is going to take the shape of custom code that would make even the Perl-iest DBA blush.

  SQL now (for dev experience) && no-SQL later (for scaling)

  no-SQL initially (for *much better* dev experience) && no-SQL later (for scaling)

Post by mulmen » Thu, Jan 16, 2025, 4:18 PM UTC

mrkeen wrote: post #57

mulmen wrote: post #49

Earlier quoted context omitted.

Yeah this is a much better initial dev experience but you still have a schema, even if you ignore it. When your objects are inconsistently shaped something has to fix them. That something is going to take the shape of custom code that would make even the Perl-iest DBA blush.

So we've shifted from: SQL now (for dev experience) && no-SQL later (for scaling) to: no-SQL initially (for *much better* dev experience) && no-SQL later (for scaling) I can get behind that. > When your objects are inconsistently shaped something has to fix them They have one schema (the class file) instead of two (the class file and the SQL migrations).