Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

dynamodbbook.com

Post reply on HN
Print view View original

61–70 of 114 posts

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#61

Post by abd12 »

arpinum wrote: post #29
maerF0x0 wrote: post #19

Earlier quoted context omitted.

> makes it difficult to scale up for bursts Can you tell me why the On Demand mode doesnt work for you?

7x the cost. I find it interesting that the DynamoDB cheer squad points out most databases only run at 10-15% utilisation and are burning money every hour. In the next breath they suggest running on demand "till it hurts" and paying AWS as if they were running at 15% utilisation.

I recommend On-Demand pricing 'until it hurts'[0], but that's because a ton of people I talk to are spending If you are at the point where you are spending over thousands of dollars a month on DynamoDB, then it does make sense to review your usage, fine-tune your capacity, set up auto-scaling, buy reserved capacity, etc. But don't waste your time doing that to save $14 a month. There are better things to do.

But it's really nice to have a database where you can set up pay-per-use, don't have to think about exhausting your resources, and have an option to back out into a cheaper billing mode if it does get expensive.

[0] - Hat tip to Jared Short for this advice & phrase

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#62

Post by abd12 »

eloff wrote: post #16
abd12 wrote: post #3

Waves Author here. Happy to answer any questions folks have about the book, about DynamoDB, or about self-publishing. NoSQL modeling is waaay different than relational modeling. I think a lot of NoSQL advice out there is pretty bad, which results in people dismissing the technology altogether. I've been working with DynamoDB for a few years now, and there's no way I'll go back. The book has been available for about a…

The biggest problem I'm aware of with DynamoDB is the hot key / partition issue[1]. Throughout is distributed evenly across nodes, you can't control how many nodes you have, so you always have a node that's hot either temporarily or permanently and so you end up having to over provision all your nodes to be able to handle that hot case, which ends up costing far more than alternatives. What's your take on this? This…

luhn responded to this one pretty well :)

Basically, most of these issues are gone. As long as you don't have extreme skew in your partition keys, you don't need to worry about throughput limits.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#63

Post by pier25 »

For my current serverless project I'm using Fauna which I think is a better option than Dynamo. You get relations, complex queries, etc. You also get authentication and authorization baked-in.

I haven't done any serious tests but I'd say on average my reads to Fauna from Cloudflare workers are 30ms. Seems a lot compared to querying a local instance of Postgres but since Fauna is distributed you end up getting much better latency on average for your worldwide users compared to a single DB in us-east-1.

Writes take longer (probably around 200-300ms on average) but considering these are replicated to all Fauna servers with ACID I'm ok with that.

I wrote a little intro to Fauna's query language which is very powerful if anyone is interested:

https://github.com/PierBover/getting-started-fauna-db-fql

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#64

Post by mmbleh »

abd12 wrote: post #58
scarface74 wrote: post #56

Earlier quoted context omitted.

Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc. If you can use Aurora Serverless, the Data API makes sense for lambda. https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

True! I'm not a huge fan of Aurora Serverless and the Data API. The scaling for Aurora Serverless is slow enough that it's not really serverless, IMO. And the Data API adds a good bit of latency and has a non-standard request & response format, so it's hard to use with existing libraries. But it's definitely an option for those that want Lambda + RDBMS. The RDS Proxy is _hopefully_ a better option in this regard but…

Differing opinion - I think RDS Proxy is the wrong approach. Adding an additional fixed cost service to enable lambda seems like an indicator of a bad architecture. In this case the better approach would likely be to just use a Fargate container which would have a similar cost and fewer moving parts.

By the time you pay a fixed cost for the proxy on top of what you already pay for the RDS server, it'd be a far simpler architecture with less moving parts to just run a Fargate container (or better yet, AWS would offer a Google Cloud Run competitor)

The Data API, while still rough around the edges, at least keeps the solution more "serverless-y". Over time it should get easier to work with as tooling improves. At the very least, it won't be more difficult to work with than DynamoDB was initially with it's different paradigm.

For services that truly require consistently low latency, lambda shouldn't be used anyway, so the added latency of the data api shouldn't be a big deal IMO.

For those reasons, I view the RDS Proxy as an ugly stopgap that enables poor architecture, whereas the Data API actually enables something new, and potentially better. So I'd much rather AWS double down on it and quickly add some improvements.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#65

Post by DVassallo »

abd12 wrote: post #60
DVassallo wrote: post #32

Earlier quoted context omitted.

If you treat DynamoDB as a DBMS, you’re going to be disappointed (for the reasons you mention). But if you think of it as a highly-durable immediately-consistent btree in the cloud, it’s amazing. DynamoDB is closer to Redis than MySQL. Amazon does it a disservice by putting it in the databases category.

Daniel, I'm a big fan of yours but disagree with this take :). It's definitely a database. The modeling principles are different, and you won't get some of the niceties you get with a RDBMS, but it still allows for flexible querying and more. S3 is not a database, but DynamoDB is :).

S3 and DDB are incredibly similar. Their fundamental operators are the same: key-value get/put and ordered list, and their consistency is roughly the same.

What differentiates DDB and S3 the most is cost and performance.

They're both highly-durable primitive data structures in the cloud, with a few extra features attached.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#66

Post by dropofwill »

philipkglass wrote: post #52
arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

Others are comparing DynamoDB to Redis and Cassandra. It has additional limitations. These are fairly clearly spelled out but maybe weren't highlighted as prominently a few years back. (I say that because I inherited an application that made heavy use of DynamoDB but turned out not to be a great fit for DDB.) - It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty val…

Note that Cassandra has similar limitations with data/throughput, but they aren't enforced or documented (because they depend on your particular setup) and your queries just fail or worse make all queries to the same node in the cluster fail (fun times with large wide rows).

The rich data types in Dynamo are quite strange, since they're basically useless for querying I'm not sure why you would use them. Maybe I'm missing something...

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#67

Post by giaour »

dropofwill wrote: post #66
philipkglass wrote: post #52

Earlier quoted context omitted.

Others are comparing DynamoDB to Redis and Cassandra. It has additional limitations. These are fairly clearly spelled out but maybe weren't highlighted as prominently a few years back. (I say that because I inherited an application that made heavy use of DynamoDB but turned out not to be a great fit for DDB.) - It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty val…

Note that Cassandra has similar limitations with data/throughput, but they aren't enforced or documented (because they depend on your particular setup) and your queries just fail or worse make all queries to the same node in the cluster fail (fun times with large wide rows). The rich data types in Dynamo are quite strange, since they're basically useless for querying I'm not sure why you would use them. Maybe I'm mis…

The rich data types can be useful for filtering, I guess? When you are running a query against a certain hash key and want the first record (sorted by range key) that meets a condition placed on a nested property of a map or whose "tags" property is a string set containing a certain member, for example.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#69

Post by TheSpiciestDev »

abarrettwilsdon wrote: post #15

I bought this a few weeks ago and am about 130 pages in. It is just stunning how much better it is learning Dynamo/NoSQL in general from this than effectively any other source. Anyone who's had to rely on AWS docs knows how face-meltingly dense they can be. I went back and refactored all my previous Dynamo work last night, and the difference was night and day. I'm planning to migrate some relational structures later…

What has this book taught you that could be applied outside DynamoDB? I'm close to buying but the price is kinda steep... if however I can take away some general NoSQL insight then I'm sold.

Edit: nevermind, I see another review elsewhere and the author replying. Though, your opinion would still be appreciated! :)

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#70

Post by dropofwill »

arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

RDS maxes out RAM at 768GiB, if we're comparing managed to managed.

If you're approaching that point, you already are going to need an analytics pipeline, a search DB, etc, because maintaining ever growing indices will kill your latency. You probably can get away with aggregations for a bit longer, but if the number of rows you aggregate is growing too, eventually you will need to come up with something and the way you do that with Dynamo off a stream isn't a bad way to go about it with MySql either.

Looking at the tables I have access to, they all come under 5ms for both read/write. This is the same ballpark as our MySql apps for similar style queries (i.e. not aggegrations).

Sadly my favorite reason to use Dynamo is political, not technical. Since it somehow is not classified as a database at my company, the DBAs don't 'own' it. So I don't have to wait 2-3 months for them to manually configure something.

Conway's law strikes again.

Post reply on HN