Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

151–160 of 225 posts

Re: DynamoDB 10 years later

#151

Post by amzn-throw »

afandian wrote: post #2

We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…

The key benefit with DDB is predictability: https://brooker.co.za/blog/2022/01/19/predictability.html

Yes, you have to learn about all these things upfront. But once you figure it out, test it, and configure it - it will work as you expect. No surprises.

Whereas Relational Databases work until they don't. A developer makes a tiny (even a no-op) change to a query or stored procedure, a different SQL plan gets chosen, and suddenly your performance/latency dramatically reduces, and you have no easy way to roll it back through source control/deployment pipelines. You have to page a DBA who has to go pull up the hood.

With services like DDB, you maintain control.

Re: DynamoDB 10 years later

#152

Post by wnolens »

otterley wrote: post #95
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

> Customers care about the outcome, not the internal process

Additionally, the business cares about the outcome, not the internal process.

Ostensibly, the business should care about process but it actually doesn't matter as long as the product is just good enough to obtain/retain customers, and the people spending the money (managers) aren't incentivized to make costs any lower than previously promised (status quo).

Re: DynamoDB 10 years later

#153

Post by sam0x17 »

salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

Re: DynamoDB 10 years later

#154

Post by greiskul »

AtlasBarfed wrote: post #134
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

So they just rolled out global replication, and I can't for the life of me figure out how they resolve write conflicts without cell timestamps or any other obvious CRDT measures. Questions were handwaved away, and the usual Amazon black box non-answers which always smells like they are hiding problems. Any ideas how this is working? It seems bolt-on and not well thought out, and I doubt they'll ever pay for Aphyr to…

From: https://aws.amazon.com/dynamodb/global-tables/

Consistency and conflict resolution

Any changes made to any item in any replica table are replicated to all the other replicas within the same global table. In a global table, a newly written item is usually propagated to all replica tables within a second. With a global table, each replica table stores the same set of data items. DynamoDB does not support partial replication of only some of the items. If applications update the same item in different Regions at about the same time, conflicts can arise. To help ensure eventual consistency, DynamoDB global tables use a last-writer-wins reconciliation between concurrent updates, in which DynamoDB makes a best effort to determine the last writer. With this conflict resolution mechanism, all replicas agree on the latest update and converge toward a state in which they all have identical data.

Re: DynamoDB 10 years later

#155

Post by [deleted] »

sam0x17 wrote: post #153
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

[deleted]

Re: DynamoDB 10 years later

#156

Post by luhn »

AtlasBarfed wrote: post #134
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

So they just rolled out global replication, and I can't for the life of me figure out how they resolve write conflicts without cell timestamps or any other obvious CRDT measures. Questions were handwaved away, and the usual Amazon black box non-answers which always smells like they are hiding problems. Any ideas how this is working? It seems bolt-on and not well thought out, and I doubt they'll ever pay for Aphyr to…

Honestly your expectations are too high. Conflict resolution is row-level last-write-wins. It's not a globally distributed database, it's just a pile of regional DynamoDB tables duct taped together... They're not going to hire Aphyr for testing because there's nothing for him to test.

Re: DynamoDB 10 years later

#157

Post by tmitchel2 »

jamesblonde wrote: post #93
300bps wrote: post #61

Earlier quoted context omitted.

In my experience, leave DynamoDB alone and it will work great. My experience agrees with yours and I'm likewise puzzled by the grandparent comment. But just a shout out to DAX (DyanmoDB Accelerator) which makes it scale through the roof: https://aws.amazon.com/dynamodb/dax/

If you add DAX you are not guaranteed to read your writes. Terrible consistency model. https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

You choose your consistency on reads. However, Dax won't help you much on a write heavy workload.

Re: DynamoDB 10 years later

#158

Post by tmitchel2 »

PaywallBuster wrote: post #78
chrisoverzero wrote: post #42

Earlier quoted context omitted.

It’s up to 27 times a day, if you time it well: “4 decreases in the first hour, and 1 decrease for each of the subsequent 1-hour windows in a day”.

gotcha, it's been awhile since I was looking at that

They upped it when they're own autoscaler needed the ability to back it down more :-/

Re: DynamoDB 10 years later

#159

Post by tmitchel2 »

DynamoDB for me is the perfect database for my serverless / graphql API. My only gripe is the limitation of items in a transaction of 25. I've had to resort to layering another transaction mgmt system on the top of it.

Re: DynamoDB 10 years later

#160

Post by 8note »

willcipriano wrote: post #21
jon-wood wrote: post #15

Earlier quoted context omitted.

This will sound flippant, but that's not what Dynamo is for. If you want to do freeform relational queries like that then put it in a relational database. Dynamo is primarily designed for high volume storage/querying on well understood data sets with a few query patterns. If you want to be able to query information on employees based on their name and city you'll need to build another index keyed on name and city (in…

Amazon has a perfect use case for this. You click on a product in the search results, that url contains a UUID, that UUID is used to search Dynamo and returns an object that has all the information on the product, from that you build the page. If what you are trying to do looks more like "Give me all the customers that live in Cuba and have spent more than $10 and have green eyes", Dynamo isn't for you. You can query…

If that's one of 12 or less query patterns you need, I can write you a simple dynamo table for it. Dynamo's limitation is that it can only support n different query patterns, and you have to hand craft an index for each one(well, sometimes you can get multiple on one index)
Post reply on HN