Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Decade of Dynamo

allthingsdistributed.com

Post reply on HN
Print view View original

11–20 of 58 posts

Re: A Decade of Dynamo

#11

Post by eropple »

rm999 wrote: post #10

DynamoDB is amazing for the right applications if you very carefully understand its limitations. Last year I built out a recommendation engine for my company; it worked well, but we wanted to make it real-time (a user would get recommendations from actions they made seconds ago, instead of hours or days ago). I planned a 4-6 week project to implement this and put it into production. Long story short: I learned about…

This is the good side of Dynamo, and it's awesome that you've had that experience.

The flip side: Dynamo gets expensive and it gets expensive quick, and being a custom API (and, indeed, a very different way to think about datastores) makes migration difficult.

It's great to use, if you understand the tradeoffs. Just make sure you understand them before you make the leap.

Re: A Decade of Dynamo

#12

Post by eropple »

sheeshkebab wrote: post #5

The thing doesn't even support a useable cross region replication. On top of that the whole read/write capacity is a joke (a painful one at that). Other than a dirty js config or a prototype store this db is useless.

I would be very, very careful of calling anything a company of such very sharp people does a "joke."

One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

Re: A Decade of Dynamo

#13

Post by jchanimal »

jchw wrote: post #7

I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.

The databases you mentioned both have strong consistency, but do not have serverless pricing models. My employer FaunaDB has a similar consistency model, but a pay-as-you-go model that requires no provisioning or capacity planning.

You can read more about our ACID transactions here: https://fauna.com/blog/consistent-transactions-in-a-globally...

Re: A Decade of Dynamo

#14

Post by sheeshkebab »

eropple wrote: post #12
sheeshkebab wrote: post #5

The thing doesn't even support a useable cross region replication. On top of that the whole read/write capacity is a joke (a painful one at that). Other than a dirty js config or a prototype store this db is useless.

I would be very, very careful of calling anything a company of such very sharp people does a "joke." One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling.

I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

Re: A Decade of Dynamo

#16

Post by p0rkbelly »

jchw wrote: post #7

I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.

I think it is a worth noting that there is a difference between Dynamo and DynamoDB. One is a powerhouse academic publication that shook up modern computer science and one an enterprise tech product based upon Dynamo.

It is the 10th anniversary of Dynamo as a CS milestone.

Re: A Decade of Dynamo

#17

Post by jchw »

jchanimal wrote: post #13
jchw wrote: post #7

I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.

The databases you mentioned both have strong consistency, but do not have serverless pricing models. My employer FaunaDB has a similar consistency model, but a pay-as-you-go model that requires no provisioning or capacity planning. You can read more about our ACID transactions here: https://fauna.com/blog/consistent-transactions-in-a-globally...

Pretty cool. Of course, the serverless version doesn't have too many regions yet, so some of the advantages of strong global consistency may be less useful. But I'll keep my eye on this regardless.

Re: A Decade of Dynamo

#18

Post by jchw »

p0rkbelly wrote: post #16
jchw wrote: post #7

I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.

I think it is a worth noting that there is a difference between Dynamo and DynamoDB. One is a powerhouse academic publication that shook up modern computer science and one an enterprise tech product based upon Dynamo. It is the 10th anniversary of Dynamo as a CS milestone.

I understand that, I'm mostly just replying to the "As we say at Amazon, it's just day one for DynamoDB" line. I do respect that Dynamo is certainly an achievement in thinking outside the box. But as for DynamoDB's place in the future of databases, I'm betting against it due to the lack of versatility for workloads that aren't purely non relational. Can't even do log style stuff due to the way sharding works :(

Re: A Decade of Dynamo

#19

Post by ryanworl »

sheeshkebab wrote: post #14
eropple wrote: post #12

Earlier quoted context omitted.

I would be very, very careful of calling anything a company of such very sharp people does a "joke." One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

You cannot replace DynamoDB with S3. S3 can not perform atomic and strongly consistent operations.

Edit: as other commenters have noted, you can perform a read after write on a new key.

Re: A Decade of Dynamo

#20

Post by peterwwillis »

When they mention companies using DynamoDB, at least one of those actually uses their own implementation of Dynamo that they wrote to work around cost and performance limitations.

The main problems faced are not the ability to scale or reach performance benchmarks or keep data safe. They are operational, and primarily problems of infrastructure complexity and management. Oh, and having developers architect and manage the operations of a really freaking huge service is a bad idea. (No offense intended - those developers don't want to be woken up in the middle of the night either)

Post reply on HN