Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

201–210 of 225 posts

Re: DynamoDB 10 years later

#201

Post by awsthro00945 »

belter wrote: post #124
awsthro00945 wrote: post #77

Earlier quoted context omitted.

I think I've seen you post something similar on r/aws about how Rick was "top DynamoDb person at AWS" (apologies if that wasn't you). I think you are overestimating Rick's "rank". I just looked him up (I had not heard of him before seeing his name mentioned on r/aws a few days ago) and he was an L7 TPM/Practice Manager in AWS's sales organization. That's not really a notably high position, and in the grand scheme of…

You never heard of Rick Houlihan? He is the 90% of DynamoDB Evangelism... At the same time you are able to this internal lookups? Do you work with DynamoDB? AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) https://youtu.be/HaEPXoXVf2k AWS re:Invent 2019: [REPEAT 1] Amazon DynamoDB deep dive: Advanced design patterns (DAT403-R1) https://youtu.be/6yqfmXiZTlM AWS re:Invent 20…

No, I haven't. There are thousands of reinvent sessions every year. I don't watch them all (I don't watch hardly any of them, and most people I know in Amazon watch a couple breakout sessions if that. Some don't even watch the keynotes). Their targeted audience is AWS customers, not internal engineers. Reinvent itself is a sales conference. If internal Amazonians want to learn about something like DDB, there are internal talks and documents given by the engineering leaders that we watch.

>At the same time you are able to this internal lookups?

I looked him up on LinkedIn. Nothing internal about it.

Re: DynamoDB 10 years later

#202

Post by leetrout »

throwdbaaway wrote: post #194
pier25 wrote: post #133

Earlier quoted context omitted.

BTW Rick Houlihan left AWS recently to work for Mongo. https://twitter.com/houlihan_rick/status/1472969503575265283 On that thread he criticizes AWS regarding DynamoDB openly. > I will always love DynamoDB, but the fact is it is losing ground fast because AWS focuses most of their resources on the half baked #builtfornopurpose database strategy. I always hated that idea, I just bit my tongue instead of saying it. > T…

Interesting. MongoDB actually came to mind while I was reading the other comment here: > No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable querying, and then they bolt on Redis to make the disjointed backend feel fast. And I'm just talking operational use cases; ignoring analy…

Not snark: did MongoDB ever go away?

I've seen it used in many places over the years.

Today I would choose JSON in Postgres before I would just jump to Monogo but it certainly serves a purpose for many shops and it is still widely used AFAIK.

I _really_ miss RethinkDB.

Re: DynamoDB 10 years later

#203

Post by awsthro00945 »

gigatexal wrote: post #170
amzn-throw wrote: post #149

Earlier quoted context omitted.

Do you expect the engineers on your team to know the top sales person at your company? This person might be responsible for the majority of evangelism and revenue for the company. Do you expect the SDEs to know about him? Again, no shot against against Rick - he is amazing, smart, technical, competent, and a deep owner. But the average SDE on the team won't know about these or watch these talks. There are too many de…

Maybe that was the problem. He cited that there was seemingly not enough effort in making DynamoDB better as evidenced by the many orthogonally very close other DBs that AWS promotes. If Rick was ears to the ground listening to customers and sending back feedback but it was falling on deaf ears that's enough ground for someone as high up and as influential and productive as him to leave. It also speaks to inner AWS t…

>It also speaks to inner AWS turmoil at least at DynamoDB.

How? Rick wasn't part of the DynamoDB service team. He wasn't an engineer, nor a manager on the team, nor even a product manager. He was a salesperson that specialized in DDB. He most likely had very little interactions, if any, with the engineering team. I don't see how him leaving speaks at all to anything about the inner workings of the engineering teams.

Rick seems cool, and after skimming some of his chats he seems really knowledgeable about the customer-facing side of DDB, and I mean absolutely no disrespect to him. But I think you're making way too many assumptions about his "rank" and "influence" within the company.

Re: DynamoDB 10 years later

#204

Post by emodendroket »

afandian wrote: post #2

We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…

Dynamo is like the opposite of fire and forget it. You really want to know your access patterns at design time.

Re: DynamoDB 10 years later

#205

Post by pier25 »

throwdbaaway wrote: post #194
pier25 wrote: post #133

Earlier quoted context omitted.

BTW Rick Houlihan left AWS recently to work for Mongo. https://twitter.com/houlihan_rick/status/1472969503575265283 On that thread he criticizes AWS regarding DynamoDB openly. > I will always love DynamoDB, but the fact is it is losing ground fast because AWS focuses most of their resources on the half baked #builtfornopurpose database strategy. I always hated that idea, I just bit my tongue instead of saying it. > T…

Interesting. MongoDB actually came to mind while I was reading the other comment here: > No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable querying, and then they bolt on Redis to make the disjointed backend feel fast. And I'm just talking operational use cases; ignoring analy…

In my circles Mongo has always been considered a bad database.

If Rick's vouching for it maybe it's time to give it a try. It must be pretty mature by now.

Re: DynamoDB 10 years later

#206

Post by marcosdumay »

geodel wrote: post #110
salil999 wrote: post #65

I can safely say that the team members working in DynamoDB are very skilled and they care deeply about the product. They really work hard and think of interesting solutions to a lot of problems that their biggest customers face which is great from a product standpoint. There are some pretty smart people working there. Engineering, however, was a disaster story. Code is horribly written and very few tests are maintain…

Very interesting! To me the takeaway is large/interesting/challenging engineering projects are pretty close to disasters generally. Some time they do become disaster actually. On the other hand if a project looks like straight up designed, neatly put into JIRA stories, and developers deliver code consistently week after week then it may be a successfully planned and delivered project. But it would mostly be doing stu…

On the other hand, if you don't go solder things and remove the duct tape from time to time, you will always come closer to a disaster, never further away.

Some projects are run like the Doomsday Clock, and nobody can get anything done. Other ones increase and decrease on complexity all the time, and those tend to catch-up to the first set quite quickly.

Re: DynamoDB 10 years later

#207

Post by leetrout »

pier25 wrote: post #205
throwdbaaway wrote: post #194

Earlier quoted context omitted.

Interesting. MongoDB actually came to mind while I was reading the other comment here: > No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable querying, and then they bolt on Redis to make the disjointed backend feel fast. And I'm just talking operational use cases; ignoring analy…

In my circles Mongo has always been considered a bad database. If Rick's vouching for it maybe it's time to give it a try. It must be pretty mature by now.

It had some operational quirks 10 years ago (allocating giant chunks of space was more of an issue that dataloss) and I've not used it directly in that many years. We lost some data during an OOM process kill but it was just twitter firehose data so not a huge deal.

Lots of good info in the response in this SO post

https://stackoverflow.com/questions/10560834/to-what-extent-...

Re: DynamoDB 10 years later

#208

Post by AtlasBarfed »

0xbadcafebee wrote: post #161
AtlasBarfed wrote: post #135

Earlier quoted context omitted.

Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?

They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality). Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoug…

Because Riak uses vector clocks instead of cell timestamps? Cassandra's ONE/QUORUM/ALL consistency levels otherwise allow tuning for tolerance of CP vs AP, don't they?

Re: DynamoDB 10 years later

#209

Post by pclmulqdq »

nexuist wrote: post #166
sam0x17 wrote: post #153

Earlier quoted context omitted.

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

They probably view it as a competitive advantage that Azure or GCP would try to copy if they figured out the "secret sauce."

Azure has a better product (Cosmos is damn good) and Google engineers have too much hubris to import something from lowly Amazon engineers.

Re: DynamoDB 10 years later

#210

Post by jamesfinlayson »

jamesblonde wrote: post #93
300bps wrote: post #61

Earlier quoted context omitted.

In my experience, leave DynamoDB alone and it will work great. My experience agrees with yours and I'm likewise puzzled by the grandparent comment. But just a shout out to DAX (DyanmoDB Accelerator) which makes it scale through the roof: https://aws.amazon.com/dynamodb/dax/

If you add DAX you are not guaranteed to read your writes. Terrible consistency model. https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

Thanks - I've seen DAX mentioned and possibly even recommended. I don't need faster DynamoDB that much.
Post reply on HN