Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

161–170 of 225 posts

Re: DynamoDB 10 years later

#161

Post by 0xbadcafebee »

AtlasBarfed wrote: post #135
0xbadcafebee wrote: post #127

Earlier quoted context omitted.

I worked at a company who re-implemented the entire Dynamo paper and API, and it was exactly the same story. Completely eliminated all my illusions about the supposed superiority of distributed systems. It was a mound of tires held together with duct tape, with a tiki torch in each tire.

Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?

They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality).

Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoughts I had about Erlang being some uber-language for distributed systems, as there were (are?) tons of bugs and missing edge cases in Riak)

Re: DynamoDB 10 years later

#162

Post by listenallyall »

otterley wrote: post #95
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

Just curious, why do you mention you work at AWS if you're just disclaiming that fact in the next sentence? Besides, nothing you stated is specific to AWS or any of its products.

Re: DynamoDB 10 years later

#163

Post by 0xbadcafebee »

wnolens wrote: post #147
0xbadcafebee wrote: post #128

Earlier quoted context omitted.

You throw bodies at it. A small bunch of people will be overworked, stressed, constantly fighting fires and struggling to fight technical debt, implement features, and keep the thing afloat. Production is always a hair away from falling over but luck and grit keeps it running. To the team it's a nightmare, to the business everything is fine.

this is the answer. source: currently being burned out on an adjacent aws team..

That sucks, man. If they won't move you to another team, just get out of there. We don't benefit by suffering for them, and they're not gonna change.

Re: DynamoDB 10 years later

#165

Post by sealjam »

listenallyall wrote: post #162
otterley wrote: post #95

Earlier quoted context omitted.

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

Just curious, why do you mention you work at AWS if you're just disclaiming that fact in the next sentence? Besides, nothing you stated is specific to AWS or any of its products.

Crudely speaking, the fact that they work at AWS means that it’s in their best interests for AWS to be perceived positively.

When this is the case it’s often nice to state this conflict of interest, so others can take your appraisal in the appropriate context.

I’m not implying anything about the post, just stating what I assume to be the reason for the disclosure.

Re: DynamoDB 10 years later

#166

Post by nexuist »

sam0x17 wrote: post #153
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

They probably view it as a competitive advantage that Azure or GCP would try to copy if they figured out the "secret sauce."

Re: DynamoDB 10 years later

#167

Post by [deleted] »

vasili111 wrote: post #68
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

>Engineering, however, was a disaster story. Code is horribly written and very few tests are maintained to make sure deployments go without issues. There was too much emphasis on deployment and getting fixes/features out over making sure it won't break anything else. It was a common scenario to release a new feature and put duct tape all around it to make sure it "works". And way too many operational issues. There ar…

[deleted]

Re: DynamoDB 10 years later

#168

Post by belter »

amzn-throw wrote: post #149
belter wrote: post #124

Earlier quoted context omitted.

You never heard of Rick Houlihan? He is the 90% of DynamoDB Evangelism... At the same time you are able to this internal lookups? Do you work with DynamoDB? AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) https://youtu.be/HaEPXoXVf2k AWS re:Invent 2019: [REPEAT 1] Amazon DynamoDB deep dive: Advanced design patterns (DAT403-R1) https://youtu.be/6yqfmXiZTlM AWS re:Invent 20…

Do you expect the engineers on your team to know the top sales person at your company? This person might be responsible for the majority of evangelism and revenue for the company. Do you expect the SDEs to know about him? Again, no shot against against Rick - he is amazing, smart, technical, competent, and a deep owner. But the average SDE on the team won't know about these or watch these talks. There are too many de…

Are you calling the person who did the core DynamoDB Technical Deep Dive sessions at reInvent, for the last 4 years in a row, a sales person?

Re: DynamoDB 10 years later

#169

Post by js4ever »

sam0x17 wrote: post #153
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

There is a compatible open source alternative here, https://www.scylladb.com/alternator/

Re: DynamoDB 10 years later

#170

Post by gigatexal »

amzn-throw wrote: post #149
belter wrote: post #124

Earlier quoted context omitted.

You never heard of Rick Houlihan? He is the 90% of DynamoDB Evangelism... At the same time you are able to this internal lookups? Do you work with DynamoDB? AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) https://youtu.be/HaEPXoXVf2k AWS re:Invent 2019: [REPEAT 1] Amazon DynamoDB deep dive: Advanced design patterns (DAT403-R1) https://youtu.be/6yqfmXiZTlM AWS re:Invent 20…

Do you expect the engineers on your team to know the top sales person at your company? This person might be responsible for the majority of evangelism and revenue for the company. Do you expect the SDEs to know about him? Again, no shot against against Rick - he is amazing, smart, technical, competent, and a deep owner. But the average SDE on the team won't know about these or watch these talks. There are too many de…

Maybe that was the problem. He cited that there was seemingly not enough effort in making DynamoDB better as evidenced by the many orthogonally very close other DBs that AWS promotes. If Rick was ears to the ground listening to customers and sending back feedback but it was falling on deaf ears that's enough ground for someone as high up and as influential and productive as him to leave. It also speaks to inner AWS turmoil at least at DynamoDB.
Post reply on HN