DynamoDB 10 years later • Hacker News Board

Re: DynamoDB 10 years later

#71

Post by badhombres » Thu, Jan 20, 2022, 2:46 PM UTC

vasili111 wrote: post #68

salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

>Engineering, however, was a disaster story. Code is horribly written and very few tests are maintained to make sure deployments go without issues. There was too much emphasis on deployment and getting fixes/features out over making sure it won't break anything else. It was a common scenario to release a new feature and put duct tape all around it to make sure it "works". And way too many operational issues. There ar…

I mean eventually enough duct tape can be solid like a tank :)

Re: DynamoDB 10 years later

#72

Post by meepmorp » Thu, Jan 20, 2022, 2:48 PM UTC

vasili111 wrote: post #68

salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

>Engineering, however, was a disaster story. Code is horribly written and very few tests are maintained to make sure deployments go without issues. There was too much emphasis on deployment and getting fixes/features out over making sure it won't break anything else. It was a common scenario to release a new feature and put duct tape all around it to make sure it "works". And way too many operational issues. There ar…

You need the really good duct tape.

Re: DynamoDB 10 years later

#73

Post by pow_pp_-1_v » Thu, Jan 20, 2022, 2:48 PM UTC

mjb wrote: post #66

tybit wrote: post #11

For anyone else expecting this to be a paper given the domain name, it’s not. It’s a non technical interview with a couple of the original papers authors. Not bad, just not as exciting as I imagine a paper detailing what they’ve learnt from a distributed systems perspective etc operating Dynamo then DynamoDB for so long now.

We don't have a paper on DynamoDB's internals (yet?), but here's a talk you might find interesting from one of the folks who built and ran DDB for a long time: https://www.youtube.com/watch?v=yvBR71D0nAQ And Doug Terry talking through the details of how DynamoDB's transaction protocol works: https://www.usenix.org/conference/fast19/presentation/terry If we did publish more about the internals of DDB, what would you b…

All of it - architecture, operational experience, best practices etc.

Re: DynamoDB 10 years later

#74

Post by jeffreygoesto » Thu, Jan 20, 2022, 2:51 PM UTC

badhombres wrote: post #71

vasili111 wrote: post #68

Earlier quoted context omitted.

>Engineering, however, was a disaster story. Code is horribly written and very few tests are maintained to make sure deployments go without issues. There was too much emphasis on deployment and getting fixes/features out over making sure it won't break anything else. It was a common scenario to release a new feature and put duct tape all around it to make sure it "works". And way too many operational issues. There ar…

I mean eventually enough duct tape can be solid like a tank :)

Or a bridge...

Re: DynamoDB 10 years later

#75

Post by _hao » Thu, Jan 20, 2022, 2:52 PM UTC

itsmemattchung wrote: post #25

victor106 wrote: post #8

We tried to implement an application on DynamoDB about 2 years ago. We really struggled with implementing adhoc queries/search. For e.g:- select * from employees where name = X and city = Y. Any improvements in DynamoDB that make it easier to implement such queries?

With DynamoDB, you can now execute SQL queries using PartiQL: https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

I'm really happy that Cosmos DB has this - https://docs.microsoft.com/en-us/azure/cosmos-db/sql/sql-que...

I haven't used DynamoDB in a couple of years, so I'd be curious to know how querying compares if anyone can share some light that has used both Cosmos and Dynamo recently.

Re: DynamoDB 10 years later

#76

Post by musingsole » Thu, Jan 20, 2022, 2:53 PM UTC

xtracto wrote: post #70

manishsharan wrote: post #53

Quite a few of the teams that were early adopters of AWS DynamoDB were not prepared for the pricing nuances that had to be taken into consideration when building their solutions.

I remember trying Dynamodb around 2015/2016: You had to specify your expected read and write throughout and you would be billed for that. At that time we had a pretty spikey traffic use case which made using dynamodb efficiently impossible

I had a similar experience, but ultimately wrote a service to monitor our workloads and request increased provisioning during spikes. You could reduce your provisioning like 10 times a day, but after that you could only increase it and would be stuck with the higher rate for a time.

And then on-demand provisioning was released and it was cheap enough to be worth simplifying our workflows.

Re: DynamoDB 10 years later

#77

Post by awsthro00945 » Thu, Jan 20, 2022, 2:54 PM UTC

gigatexal wrote: post #28

vslira wrote: post #16

Earlier quoted context omitted.

Man I love Rick’s talks as much as anyone but let’s be real, he likely left AWS not for his love of first class geographical indexes but because Mongo offered a giant pile of money for him to evangelize their tech. Though I have no doubts that he actually had a lot of reservations around Dynamo’s DX before, he likely has some around mongodb but those won’t be the bulk of his content

At his rank at AWS I don’t know if money was such an issue. He strikes me as a person who cares deeply about the underlying tech. But I have no idea one way or the other.

I think I've seen you post something similar on r/aws about how Rick was "top DynamoDb person at AWS" (apologies if that wasn't you). I think you are overestimating Rick's "rank".

I just looked him up (I had not heard of him before seeing his name mentioned on r/aws a few days ago) and he was an L7 TPM/Practice Manager in AWS's sales organization. That's not really a notably high position, and in the grand scheme of Amazon pay scales, isn't that high up. An L7 TPM gets paid about the same as, or sometimes less than, an L6 software dev (L6 is "senior", which is ~5-10 years of experience).

Also, him being in the sales org means he had practically nothing to do with the engineering of the service. AWS Sales is a revolving door of people. I mean no offense towards Rick (again, I didn't know him or even know of him before I read his name in a comment a few days ago), but I would not read anything at all into the fact that an L7 Sales TPM left for another company.

Re: DynamoDB 10 years later

#78

Post by PaywallBuster » Thu, Jan 20, 2022, 2:57 PM UTC

chrisoverzero wrote: post #42

PaywallBuster wrote: post #37

Earlier quoted context omitted.

scaling down is limited to 4x a day

It’s up to 27 times a day, if you time it well: “4 decreases in the first hour, and 1 decrease for each of the subsequent 1-hour windows in a day”.

gotcha, it's been awhile since I was looking at that

Re: DynamoDB 10 years later

#79

Post by Tehnix » Thu, Jan 20, 2022, 2:59 PM UTC

afandian wrote: post #2

We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…

A lot of things that used to be a concern (hot partitions, etc) are not a concern anymore and most have been solved these days :)

Put it on on-demand pricing (it'll be better and cheaper for you most likely), and it will handle any load you throw at it. Can you get it to throttle? Sure, if you absolutely blast it without ever having had that high of a need before (and it can actually be avoided[0]).

You will need to understand how to model things for the NoSQL paradigm that DynamoDB uses, but that's a question of familiarity and not much else (you didn't magically know SQL either).

My experience comes from scaling DynamoDB in production for several years, handling both massive IoT data ingestion in it as well as the user data as well. We were able to replace all things we thought we would need a relational database for, completely.

My comparison between a traditional RDS setup: - DynamoDB issues? 0. Seriously. Only thing you need to monitor is billing. - RDS? Oh boy, need to provision for peak capacity, need to monitor replica lags, need to monitor the Replicas themselves, constant monitoring and scaling of IOPS, suddenly queries get slow as data increases, worrying about indexes and the data size, and much more...

[0]: https://theburningmonk.com/2019/03/understanding-the-scaling...

Re: DynamoDB 10 years later

#80

Post by grncdr » Thu, Jan 20, 2022, 3:00 PM UTC

jerf wrote: post #59

redwood wrote: post #18

Earlier quoted context omitted.

Your example really summarizes the challenge with the AWS paradigm: namely that they want you to believe that the thing to do is to spread the the backend of your application across a large number of distinct data systems. No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable quer…

It seems to me that what this is saying is that storage has become so cheap that if another database provides even slight advantages over another for some workload it is likely to be deployed and have all the data copied over to it. HN entrepreneurs take note, this also suggests to me that there may be a market for a database (or a "metadatabase") that takes care of this for you. I'd love to be able to have a "relati…

I think there was a project like this a few years ago (wrapping a relational DB + ElasticSearch into one box) and I thought it was CrateDB, but from looking at their current website I think I'm misremembering.

The concept didn't appeal to me very much then, so I never looked into it further.

---

To address your larger point, I think Postgres has a better chance of absorbing other datastores (via FDW and/or custom index types) and updating them in sync with it's own transactions (as far as those databases support some sort of atomic swap operation) than a new contender has of getting near Postgres' level of reliability and feature richness.