Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

dynamodbbook.com

Post reply on HN
Print view View original

51–60 of 114 posts

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#51

Post by abd12 »

arpinum wrote: post #50
abd12 wrote: post #37

Earlier quoted context omitted.

Fair enough! I think that's a reasonable position. IMO, there are two times you should absolutely default to DynamoDB: - Very high scale workloads, due to its scaling characteristics - Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. You can use DynamoDB for almost all OLTP workloads, but outside of those two categories, I won't fault you for cho…

> Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. This is only true for AWS. Azure functions share resources and don't have this issue. The speed is actually quite sad. Its 5-10x slower than my other databases at p95, and I can't throw money at the problem on the write side. Reads I can use DAX, but then there goes consistency.

Good point! I would usually not recommend using a database from a different cloud provider just because of different hassles around permissions, connections, etc.

I've never found the speed an issue, but YMMV. To me, the best thing is that you won't see speed degradation as you scale. With a relational database, your joins will get slower and slower as the size of your database grows. With DynamoDB, it's basically the same at 1GB as it is at 10TB.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#52

Post by philipkglass »

arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

Others are comparing DynamoDB to Redis and Cassandra. It has additional limitations. These are fairly clearly spelled out but maybe weren't highlighted as prominently a few years back. (I say that because I inherited an application that made heavy use of DynamoDB but turned out not to be a great fit for DDB.)

- It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty values.

- You can store a maximum 400 KB data in one row.

- You can get a maximum of 1 MB data returned in a single query.

So it's mostly good for high-data-throughput applications, and then only if your high data throughput consists of large numbers of small records, processed a few at a time. This surely describes an important class of workloads. You may suffer if your workload isn't in this class.

Another annoyance is that (in my experience) one of the most common errors you will encounter is ProvisionedThroughputExceededException, when your workload changes faster than the auto-scaling. Until last year you couldn't test this scenario offline with the DynamoDB Local service because DynamoDB Local didn't implement capacity limits.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#53

Post by StreamBright »

arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

We used to run Riak for Dynamo like workloads very efficiently, 30ms p50 insertion time.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#54

Post by tinkertamper »

After using Dynamo for 2 years now the biggest problem I’ve seen thus far is the pretty extreme expectations it puts on your application code to manage things that have traditionally been considered the responsibility of the data store. We found it was a bit onerous to ensure all facets of modeling/validation/indexing were into consideration when writing that layer of the application. To address the constant bootstrapping you either end up with a crap ton of utilities that form indexes or create updateExpression strings, etc, or you end up constantly reinventing the wheel.

The JS landscape for Dynamo is a bit bare, notable options all largely ignore the indexing principles that are the real draw of Dynamo. This heartburn caused me to sit down and write a library myself (https://github.com/tywalch/electrodb) that allows you focus on the models and relationships while taking care of all the little pitfalls and “hacky” tricks inherent in single table design.

Alex’s book covers all these things and I honestly wish I had had it sooner before having to learn via foot shooting. It’s pricey but if you have a need for Dynamo on your project it really pays off knowing you’re swimming with the current, and Alex definitely gets you there.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#56

Post by scarface74 »

abd12 wrote: post #37
arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

Fair enough! I think that's a reasonable position. IMO, there are two times you should absolutely default to DynamoDB: - Very high scale workloads, due to its scaling characteristics - Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. You can use DynamoDB for almost all OLTP workloads, but outside of those two categories, I won't fault you for cho…

Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc.

If you can use Aurora Serverless, the Data API makes sense for lambda.

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#57

Post by scarface74 »

bufferoverflow wrote: post #49
abd12 wrote: post #37

Earlier quoted context omitted.

Fair enough! I think that's a reasonable position. IMO, there are two times you should absolutely default to DynamoDB: - Very high scale workloads, due to its scaling characteristics - Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. You can use DynamoDB for almost all OLTP workloads, but outside of those two categories, I won't fault you for cho…

There's a third use: if you want a free ride, AWS free tier for DynamoDB is quite nice, enough to run a decent dynamic website.

Especially combined with the always free tier of lambda....

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#58

Post by abd12 »

scarface74 wrote: post #56
abd12 wrote: post #37

Earlier quoted context omitted.

Fair enough! I think that's a reasonable position. IMO, there are two times you should absolutely default to DynamoDB: - Very high scale workloads, due to its scaling characteristics - Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. You can use DynamoDB for almost all OLTP workloads, but outside of those two categories, I won't fault you for cho…

Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. If you can use Aurora Serverless, the Data API makes sense for lambda. https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

True! I'm not a huge fan of Aurora Serverless and the Data API. The scaling for Aurora Serverless is slow enough that it's not really serverless, IMO. And the Data API adds a good bit of latency and has a non-standard request & response format, so it's hard to use with existing libraries. But it's definitely an option for those that want Lambda + RDBMS.

The RDS Proxy is _hopefully_ a better option in this regard but still early.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#59

Post by abd12 »

agustif wrote: post #55

Can some knowledge be transferred to other NoSQL flavours like mongo or is the book heavily specific about DynamoDB?

All the examples are specific to DynamoDB and use DynamoDB features.

That said, the principles apply pretty well to other popular NoSQL databases, especially MongoDB and Cassandra. There will be some slight differences -- MongoDB allows better nesting and querying on nested objects -- but it's broadly the same. If you want to model NoSQL for scale, you need to use these general patterns.

If you want to check it out but find out it doesn't work for you, just let me know. I've got a 100% money-back guarantee with no questions asked if you don't like it.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#60

Post by abd12 »

DVassallo wrote: post #32
arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

If you treat DynamoDB as a DBMS, you’re going to be disappointed (for the reasons you mention). But if you think of it as a highly-durable immediately-consistent btree in the cloud, it’s amazing. DynamoDB is closer to Redis than MySQL. Amazon does it a disservice by putting it in the databases category.

Daniel, I'm a big fan of yours but disagree with this take :).

It's definitely a database. The modeling principles are different, and you won't get some of the niceties you get with a RDBMS, but it still allows for flexible querying and more.

S3 is not a database, but DynamoDB is :).

Post reply on HN