Live data from Hacker News

A Decade of Dynamo

allthingsdistributed.com

41–50 of 58 posts

Re: A Decade of Dynamo

#41

Congrats to aws on the impact DynamoDb has had on the ecosystem & industry. The article does make it seems like DynamoDb was the first to publish a unique noSQL architecture. Is this true?

No, couchDb is older than DynamoDB, and according to wikipedia there's been nosql databases since the 60s

Re: A Decade of Dynamo

#42
post #22

Earlier quoted context omitted.

>Dynamo gets expensive and it gets expensive quick, DynamoDB's pricing scales sublinearly with volume; if it starts getting expensive it was an initial misuse of DynamoDB that got obvious with scale. There are a lot of factors that go into whether you should use DynamoDB and how you implement it. I recommend anyone who is considering using it very carefully understand this page first: http://docs.aws.amazon.com/amazo…

This is how enterprise developers use the database, sometimes: https://thedailywtf.com/articles/The-Query-of-Despair You do this on your own server, slowness and bad performance are the result (but it may never, or very rarely get called). You do it on dynamo, a $10k bill may be the result.

Formatting is obviously an issue but the query length isn't a sign for bad programming. I've created some queries that probably have nearly that length. They filter, join and prepare data in ways that would probably need as much LOC in other languages while running much faster (as the database server can better optimise for structures). In this case there obviously was a problem but it could've been that someone changed an index in one table which then affected the query.

But I agree, I also don't like running databases billed by load. The risk of costly bugs is just too high.

Re: A Decade of Dynamo

#43
post #12

The thing doesn't even support a useable cross region replication. On top of that the whole read/write capacity is a joke (a painful one at that). Other than a dirty js config or a prototype store this db is useless.

I would be very, very careful of calling anything a company of such very sharp people does a "joke." One of my prior gigs was pushing a billion data points a day through DynamoDB without it breaking a sweat. We were paying for it, too--but it was there and it worked.

While I don't think Dynamo is a joke, pushing 1bn data points per day through a database system is not much. Given you have enough storage, a postgresql instance on a $50/month dedicated server can achieve that easily (from experience). You have to pay more attention to the data structure (unless you just put everything in jsonb columns) but will probably save 90% on operational costs.

Re: A Decade of Dynamo

#44

I was at AWS from 2008 to 2016. Werner Vogels, Amazon's CTO (yep, not just AWS', but Amazon's, as he had to point out numerous times) has been one of the most talented, humble and generous senior exec I've ever met in my life. Lots of good memories of time spent with him, and one of the sad aspects for me of leaving Amazon. His blog writings are really interesting. If you haven't already, I suggest you search the arc…

I have always tremendously admired that guy; what a job he has! He is literally responsible for about half of the shit on the internet not going down (and also running the biggest internet shopping mall and making sure a bunch of crappy speakers can talk, but both of those are straightforward by comparison IMO).

Can you even imagine? I want to know what his direct report structure looks like.

Re: A Decade of Dynamo

#45
post #7

I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.

Capacity provisioning was the most annoying aspect of using DynamoDB when I last worked with it, but in the last couple of months they've added autoscaling[1] which would seem to address that major bugbear (with the caveat that I haven't had reason to test it out yet ;)

[1]: https://aws.amazon.com/blogs/aws/new-auto-scaling-for-amazon...

Re: A Decade of Dynamo

#46

Congrats to aws on the impact DynamoDb has had on the ecosystem & industry. The article does make it seems like DynamoDb was the first to publish a unique noSQL architecture. Is this true?

Lotus Notes was first. Distributed, replicated key value store.

Actually I'm not right: http://blog.knuthaugen.no/2010/03/a-brief-history-of-nosql.h...

Re: A Decade of Dynamo

#47
post #34

Need a library sdk for a Dynamo Sync feature to allow easy development of offline mobile apps. Similar function to Cognito Sync. Also hope that AWS will release a serverless SQL db. And cheaper price.

> Also hope that AWS will release a serverless SQL db. You mean like RDS?

RDS is serverful not serverless.

Re: A Decade of Dynamo

#48
post #38

> The Dynamo paper was well-received and served as a catalyst to create the category of distributed database technologies commonly known today as "NoSQL." No, sorry, it was Memcached and Bigtable paper that popularized "NoSQL" term. Although there were many NoSQL databases tracing way back to 60s [1], those were the ones that "served as catalyst" for the term "NoSQL". [1] http://blog.knuthaugen.no/2010/03/a-brief-his…

Dynamo was certainly one of the products that spiked interest in the “NoSQL” datastore category.

The phrasing “served as a catalyst” seems right — it doesn’t imply the only catalyst.

Re: A Decade of Dynamo

#49
post #9

I don't know why amazon is so taken with Dynamodb. I find it to be incredibly unintuitive and lacking real world application, requiring applications to perform gymnastics to work with it.

I've found just the opposite actually. While it's far from perfect, it has been amazing for rapidly standing up new apps (especially prototypes). We've used quite a few different strategies and found it to be flexible and performant. The only downside is we do find ourselves sometimes implementing relational DB functionality at the application level to compensate for Dynamo DB's "flexibility." Postgres is still the g…

[deleted]

Re: A Decade of Dynamo

#50
post #9

I don't know why amazon is so taken with Dynamodb. I find it to be incredibly unintuitive and lacking real world application, requiring applications to perform gymnastics to work with it.

I've found just the opposite actually. While it's far from perfect, it has been amazing for rapidly standing up new apps (especially prototypes). We've used quite a few different strategies and found it to be flexible and performant. The only downside is we do find ourselves sometimes implementing relational DB functionality at the application level to compensate for Dynamo DB's "flexibility." Postgres is still the g…

>> we do find ourselves sometimes implementing relational DB functionality at the application level to compensate for Dynamo DB's "flexibility."

Yep, this is A-grade crazy, and exactly my point. I would question if it's "sometimes", or "actually almost all the time, now that we think about it, there's not much that we CAN do with DynamoDB without writing application level database functionality."

Post reply on HN