Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

171–180 of 225 posts

Re: DynamoDB 10 years later

#171

Post by spookthesunset »

AtlasBarfed wrote: post #135
0xbadcafebee wrote: post #127

Earlier quoted context omitted.

I worked at a company who re-implemented the entire Dynamo paper and API, and it was exactly the same story. Completely eliminated all my illusions about the supposed superiority of distributed systems. It was a mound of tires held together with duct tape, with a tiki torch in each tire.

Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?

> Why did they not just run cassandra?

Not Invented Here can run very deep in some branches of an organization. Depending on how engineering performance evaluations work, writing a homebrew database could totally be something that aligns with the company incentives. It might not make a single bit of sense from a business standpoint but hey, if the company rewards such behavior don't be surprised when engineers flush millions down the tube "innovating" a brand new wheel.

Re: DynamoDB 10 years later

#172

Post by blowski »

nexuist wrote: post #166
sam0x17 wrote: post #153

Earlier quoted context omitted.

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

They probably view it as a competitive advantage that Azure or GCP would try to copy if they figured out the "secret sauce."

Azure has Cosmos which is arguably better than DynamoDB for a lot of use cases.

Re: DynamoDB 10 years later

#173

Post by [deleted] »

nathanfig wrote: post #103
didip wrote: post #101

To be honest, as a customer, it is hard for me to justify using DynamoDB. Some of this criticism can be out of date: 1. DynamoDB is not as convenient. There are a bit too many dials to turn. 2. DynamoDB does not have a SQL facade on top. 3. DynamoDB is proprietary, I believe there's no OSS API equivalent if you want to migrate out. 4. DynamoDB was kind of expensive. But it has been a while since I last check the pric…

That’s surprising to me, I consider DynamoDB to be far simpler than any relational DB including the alternatives you list

[deleted]

Re: DynamoDB 10 years later

#174

Post by Zimnx »

didip wrote: post #101

To be honest, as a customer, it is hard for me to justify using DynamoDB. Some of this criticism can be out of date: 1. DynamoDB is not as convenient. There are a bit too many dials to turn. 2. DynamoDB does not have a SQL facade on top. 3. DynamoDB is proprietary, I believe there's no OSS API equivalent if you want to migrate out. 4. DynamoDB was kind of expensive. But it has been a while since I last check the pric…

Ad 3. Lot of people don't know about it, but there's a open source, free, and DynamoDB compatibile databse called ScyllaDB - API it's called Alternator to be specific.

Re: DynamoDB 10 years later

#175

Post by staticassertion »

nexuist wrote: post #166
sam0x17 wrote: post #153

Earlier quoted context omitted.

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

They probably view it as a competitive advantage that Azure or GCP would try to copy if they figured out the "secret sauce."

I kinda doubt it. It's probably just that open sourcing it won't provide much utility (I bet lots of code is aws specific) and just adds a new maintenance burden for them.

Re: DynamoDB 10 years later

#176

Post by staticassertion »

0xbadcafebee wrote: post #161
AtlasBarfed wrote: post #135

Earlier quoted context omitted.

Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?

They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality). Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoug…

Erlang gives you great primitives for building reliable protocols, but they're just primitives, and there are tons of footguns since building protocols is hard.

Re: DynamoDB 10 years later

#177

Post by wnolens »

ddoolin wrote: post #117

As a developer, I really have a love-hate relationship with Dynamo. I love how fast and easy it is to setup and get rolling. The partitioning scheme came off as confusing and opaque but I think that says more about Amazon's documentation than the scheme itself. I do not like that there's no really third party tooling integration to be able to query. Their UI in the console is _so freaking terrible_ yet you have no ot…

Yes the console UI is mind numbing, and was just made 10x worse in a recent redesign.

But i like the python boto3 library https://boto3.amazonaws.com/v1/documentation/api/latest/guid...

Build yourself a few wrappers to make querying more convenient, and i query straight from a python repl pretty effectively

Re: DynamoDB 10 years later

#178

Post by sam0x17 »

nexuist wrote: post #166
sam0x17 wrote: post #153

Earlier quoted context omitted.

It's a shame they don't open source it. It's funny too, being AWS they really don't have to worry about AWS running a cheaper service, so at that point why not open source it.

They probably view it as a competitive advantage that Azure or GCP would try to copy if they figured out the "secret sauce."

Azure has Cosmos and Google has Datastore. They would never.

Re: DynamoDB 10 years later

#179

Post by sokoloff »

listenallyall wrote: post #162
otterley wrote: post #95

Earlier quoted context omitted.

Customers care about the outcome, not the internal process. Besides, I’ve never worked at any sizable company in my 20+-year-long career where I didn’t conclude, “it’s a miracle this garbage works at all.” Enjoy the sausage, but if you have a weak stomach, don’t watch how it’s made. (I work for AWS but not on the DynamoDB team and I have no first-hand knowledge of the above claim. Opinions are my own and not those of…

Just curious, why do you mention you work at AWS if you're just disclaiming that fact in the next sentence? Besides, nothing you stated is specific to AWS or any of its products.

I don't work at Amazon, but our company's social media policy requires us to be transparent about a possible conflict of interest when speaking about things "close to" our company/our position in the industry and also need to be clear about whether we're speaking in an official capacity or in a personal capacity.

This is designed to reduce the chances of eager employees going out and astro-turfing or otherwise acting in trust-damaging ways while thinking they're "helping".

Re: DynamoDB 10 years later

#180

Post by SomeCallMeTim »

300bps wrote: post #61
sebastialonso wrote: post #55

Earlier quoted context omitted.

> they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable querying, and then they bolt on Redis to make the disjointed backend feel fast. This made my head explode. Why would you explicitly join two systems made to solve different issues together? This sounds rather like a lack of architectural vision. Postgre…

In my experience, leave DynamoDB alone and it will work great. My experience agrees with yours and I'm likewise puzzled by the grandparent comment. But just a shout out to DAX (DyanmoDB Accelerator) which makes it scale through the roof: https://aws.amazon.com/dynamodb/dax/

In my experience, NoSQL is almost never the right answer.

And DynamoDB is worse than most.

My prediction is that the future is in scalable SQL; CockroachDB or Yugabase or similar.

NoSQL actually causes more problems than it solves, in my experience.

Post reply on HN