Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Decade of Dynamo

allthingsdistributed.com

Post reply on HN
Print view View original

21–30 of 58 posts

Re: A Decade of Dynamo

#21

Post by maldeh »

sheeshkebab wrote: post #14
eropple wrote: post #12

Earlier quoted context omitted.

I would be very, very careful of calling anything a company of such very sharp people does a "joke." One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

> Anything that can go to dynamo, can go to s3, especially at that volume.

Please think very carefully before architecting your app with S3 as a makeshift-database. S3 would be a valid option if you don't care about millisecond latency; don't require safe updates; never expect your application to scale past 100 requests per second; and don't have multiple query patterns for the same data (unless you're okay with several redundant copies of the same dataset). Consider just about any other database solution if this does not hold true.

Re: A Decade of Dynamo

#22

Post by rm999 »

eropple wrote: post #11
rm999 wrote: post #10

DynamoDB is amazing for the right applications if you very carefully understand its limitations. Last year I built out a recommendation engine for my company; it worked well, but we wanted to make it real-time (a user would get recommendations from actions they made seconds ago, instead of hours or days ago). I planned a 4-6 week project to implement this and put it into production. Long story short: I learned about…

This is the good side of Dynamo, and it's awesome that you've had that experience. The flip side: Dynamo gets expensive and it gets expensive quick , and being a custom API (and, indeed, a very different way to think about datastores) makes migration difficult. It's great to use, if you understand the tradeoffs. Just make sure you understand them before you make the leap.

>Dynamo gets expensive and it gets expensive quick,

DynamoDB's pricing scales sublinearly with volume; if it starts getting expensive it was an initial misuse of DynamoDB that got obvious with scale. There are a lot of factors that go into whether you should use DynamoDB and how you implement it. I recommend anyone who is considering using it very carefully understand this page first: http://docs.aws.amazon.com/amazondynamodb/latest/developergu...

Re: A Decade of Dynamo

#23

Post by monkmartinez »

ryanworl wrote: post #19
sheeshkebab wrote: post #14

Earlier quoted context omitted.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

You cannot replace DynamoDB with S3. S3 can not perform atomic and strongly consistent operations. Edit: as other commenters have noted, you can perform a read after write on a new key.

I am pretty sure S3 is atomic. That is, you can't get a transitory state. Your object either updated/PUT or it didn't AKA read after write consistency.

Re: A Decade of Dynamo

#24

Post by andrewguenther »

sheeshkebab wrote: post #14
eropple wrote: post #12

Earlier quoted context omitted.

I would be very, very careful of calling anything a company of such very sharp people does a "joke." One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

If you think you can replace Dynamo with S3, your application didn't need Dynamo in the first place. There appears to be some severe Dunning-Kruger at work here.

Re: A Decade of Dynamo

#25

Post by andrewguenther »

ryanworl wrote: post #19
sheeshkebab wrote: post #14

Earlier quoted context omitted.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

You cannot replace DynamoDB with S3. S3 can not perform atomic and strongly consistent operations. Edit: as other commenters have noted, you can perform a read after write on a new key.

S3 is atomic, but you are correct that it is not strongly consistent.

Re: A Decade of Dynamo

#26

Post by [deleted] »

monkmartinez wrote: post #23
ryanworl wrote: post #19

Earlier quoted context omitted.

You cannot replace DynamoDB with S3. S3 can not perform atomic and strongly consistent operations. Edit: as other commenters have noted, you can perform a read after write on a new key.

I am pretty sure S3 is atomic. That is, you can't get a transitory state. Your object either updated/PUT or it didn't AKA read after write consistency.

[deleted]

Re: A Decade of Dynamo

#27

Post by saurik »

maldeh wrote: post #21
sheeshkebab wrote: post #14

Earlier quoted context omitted.

Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.

> Anything that can go to dynamo, can go to s3, especially at that volume. Please think very carefully before architecting your app with S3 as a makeshift-database. S3 would be a valid option if you don't care about millisecond latency; don't require safe updates; never expect your application to scale past 100 requests per second; and don't have multiple query patterns for the same data (unless you're okay with seve…

I am going to say that I did this, and will add "are willing to pay some insane amount of money to store and manipulate your data" (this was a mistake that cost me at least one if not two hundred thousand dollars).

Re: A Decade of Dynamo

#28

Post by freedomben »

eropple wrote: post #11
rm999 wrote: post #10

DynamoDB is amazing for the right applications if you very carefully understand its limitations. Last year I built out a recommendation engine for my company; it worked well, but we wanted to make it real-time (a user would get recommendations from actions they made seconds ago, instead of hours or days ago). I planned a 4-6 week project to implement this and put it into production. Long story short: I learned about…

This is the good side of Dynamo, and it's awesome that you've had that experience. The flip side: Dynamo gets expensive and it gets expensive quick , and being a custom API (and, indeed, a very different way to think about datastores) makes migration difficult. It's great to use, if you understand the tradeoffs. Just make sure you understand them before you make the leap.

On the flip side tho, Dynamo can be cheap for small apps. The cost to provision an RDS instance with backup can easily be 10x to 20x what the same load would take on Dynamo.

Of course as you scale this an be less true, so it's all in the application.

Re: A Decade of Dynamo

#29

Post by freedomben »

fiokoden wrote: post #9

I don't know why amazon is so taken with Dynamodb. I find it to be incredibly unintuitive and lacking real world application, requiring applications to perform gymnastics to work with it.

I've found just the opposite actually. While it's far from perfect, it has been amazing for rapidly standing up new apps (especially prototypes). We've used quite a few different strategies and found it to be flexible and performant.

The only downside is we do find ourselves sometimes implementing relational DB functionality at the application level to compensate for Dynamo DB's "flexibility." Postgres is still the go-to for data that is relational in nature. But man, letting Amazon worry about hosting and scaling is also pretty awesome...

Re: A Decade of Dynamo

#30

Post by freedomben »

sheeshkebab wrote: post #5

The thing doesn't even support a useable cross region replication. On top of that the whole read/write capacity is a joke (a painful one at that). Other than a dirty js config or a prototype store this db is useless.

I don't know if your comment was just a troll or not, but we've build serious apps that use Dynamo DB as our store. It's been exactly what we needed, and we've got millions and millions of records across quite a few tables. There's certainly pros and cons to each database, but Dynamo can really shine if you take the time to understand it.
Post reply on HN