Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

141–150 of 225 posts

Re: DynamoDB 10 years later

#141

Post by chrisfosterelli »

otterley wrote: post #95
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

> Customers care about the outcome, not the internal process.

This is true though there's only so much technical debt and internal process chaos you can create before it affects the outcome. It's a leading indicator, so by the time customers are feeling that pain you've got a lot of work in front of you before you can turn it around, if at all, and customers are not going to be happy for that duration.

Technical debt is not something to completely defeat or completely ignore, instead it's a tradeoff to manage.

Re: DynamoDB 10 years later

#142

Post by jugg1es »

rmbyrro wrote: post #120
jugg1es wrote: post #48

Earlier quoted context omitted.

I do not recommend starting off with a decision to use DynamoDB before you have worked with it directly for some time to understand it. You could spend months trying to shoehorn your use case into it before realizing you made a mistake. That said, DynamoDB can be incredibly powerful and inexpensive tool if used right.

I think this can be said about any technology, really...

Yea, probably, but it is especially true for DynamoDB because it can initially appear as though your use cases are all supported but that is only because you haven't internalized how it works yet. By the time you realize you made a mistake, you are way too far in the weeds and have to start over from scratch. I would venture that more than 50% of DynamoDB users have had this happen to them early on. Anecdotally, just look at the comments on this post. There are so many horror stories with DynamoDB, but they're basically all people who decided to use it before they really understood it.

Re: DynamoDB 10 years later

#143

Post by Jach »

cmollis wrote: post #91

we use DynamoDB like a big hash table of s3 file locations.. we look up these locations via a key (at the time, it sounded like a pretty good use-case for it). I suppose we could have used some other managed redis or memcached thing, but being an AWS shop, it was, and is, pretty useful. I have to say, it's been pretty effortless to configure.. read/write units are really the only thing we've had to configure (other t…

I remember talking to someone who was playing with AWS stuff for the first time and they had a similar architecture, using Dynamo for a lookup store. It still seems a bit odd to me though. It's been a long time since I've worked with the S3 API, so maybe it just doesn't support the same sort of thing, but wouldn't it be nicer to just query S3 with some key and get back either the path/URL to render a link, or the content itself? Why the Dynamo intermediary? (And on the other side, if you don't need to render a link to serve the content, why not use Dynamo as the actual document store and skip S3? Storage cost?)

Re: DynamoDB 10 years later

#144

Post by bastardoperator »

jeffreygoesto wrote: post #74
badhombres wrote: post #71

Earlier quoted context omitted.

I mean eventually enough duct tape can be solid like a tank :)

Or a bridge... https://www.popularmechanics.com/science/a5732/mythbusters-b...

Or a boat...

https://flexsealproducts.com/products/flex-tape

Re: DynamoDB 10 years later

#145

Post by jedberg »

afandian wrote: post #2

We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…

> Is the dream of a self-managing, fire-and-forget key value database completely naive?

It's not, if you plan it right. Learn about single table design for DynamoDB before you start. There are a lot of good resources from Amazon and the community.

Here is a very accessible video from the community:

https://www.youtube.com/watch?v=BnDKD_Zv0og

Here is a video from Rick Houlihan, a senior leader from AWS who basically helps companies convert to single table design:

https://www.youtube.com/watch?v=KYy8X8t4MB8

And a good book on the topic:

https://www.dynamodbbook.com

If you use single table design, you can turn on all of the auto-tuning features of DynamoDB and they will work as expected and get better and more efficient with more data.

Some people worry that this breaks the cardinal rule of microservices: One database per service. But the actual rule is never have one service directly access the data of another, always use the API. So as long as your services use different keyspaces and never access each other's data, it can still work (but does require extra discipline).

Re: DynamoDB 10 years later

#146

Post by deanCommie »

salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

If the developers are happy about the code and testing quality of a project, then you waited too long to ship.

If the customers don't have any feedback or missed feature asks at launch, you waited too long to ship.

You know who has great internal code and test quality? Google. Which is why Google doesn't ship. They're a wealth distribution charity for talented engineers. And their competitive advantage is that they lure talented people away from other companies where they might actually ship something and compete with Google, to instead park them, distract them with toys, beer kegs, readability reviews, and monorepo upgrades.

Re: DynamoDB 10 years later

#147

Post by wnolens »

0xbadcafebee wrote: post #128
vasili111 wrote: post #68

Earlier quoted context omitted.

>Engineering, however, was a disaster story. Code is horribly written and very few tests are maintained to make sure deployments go without issues. There was too much emphasis on deployment and getting fixes/features out over making sure it won't break anything else. It was a common scenario to release a new feature and put duct tape all around it to make sure it "works". And way too many operational issues. There ar…

You throw bodies at it. A small bunch of people will be overworked, stressed, constantly fighting fires and struggling to fight technical debt, implement features, and keep the thing afloat. Production is always a hair away from falling over but luck and grit keeps it running. To the team it's a nightmare, to the business everything is fine.

this is the answer.

source: currently being burned out on an adjacent aws team..

Re: DynamoDB 10 years later

#149

Post by amzn-throw »

belter wrote: post #124
awsthro00945 wrote: post #77

Earlier quoted context omitted.

I think I've seen you post something similar on r/aws about how Rick was "top DynamoDb person at AWS" (apologies if that wasn't you). I think you are overestimating Rick's "rank". I just looked him up (I had not heard of him before seeing his name mentioned on r/aws a few days ago) and he was an L7 TPM/Practice Manager in AWS's sales organization. That's not really a notably high position, and in the grand scheme of…

You never heard of Rick Houlihan? He is the 90% of DynamoDB Evangelism... At the same time you are able to this internal lookups? Do you work with DynamoDB? AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) https://youtu.be/HaEPXoXVf2k AWS re:Invent 2019: [REPEAT 1] Amazon DynamoDB deep dive: Advanced design patterns (DAT403-R1) https://youtu.be/6yqfmXiZTlM AWS re:Invent 20…

Do you expect the engineers on your team to know the top sales person at your company?

This person might be responsible for the majority of evangelism and revenue for the company. Do you expect the SDEs to know about him?

Again, no shot against against Rick - he is amazing, smart, technical, competent, and a deep owner.

But the average SDE on the team won't know about these or watch these talks. There are too many deep internal engineering challenges to solve.

Re: DynamoDB 10 years later

#150

Post by jsdalton »

chrisfosterelli wrote: post #141
otterley wrote: post #95

Earlier quoted context omitted.

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

> Customers care about the outcome, not the internal process. This is true though there's only so much technical debt and internal process chaos you can create before it affects the outcome. It's a leading indicator, so by the time customers are feeling that pain you've got a lot of work in front of you before you can turn it around, if at all, and customers are not going to be happy for that duration. Technical debt…

This article from Martin Fowler explores your point in greater depth. It's a good read: https://martinfowler.com/articles/is-quality-worth-cost.html

One concrete problem with technical debt the article highlights is it that negatively impacts the time to deliver new features. Customers today usually expect not only a great initial feature set from a product, but also a steady stream of improvements and growth, along with responsiveness to feedback and pain points.

Post reply on HN