Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

dynamodbbook.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#81

Post by glkz »

dropofwill wrote: post #66
philipkglass wrote: post #52

Earlier quoted context omitted.

Others are comparing DynamoDB to Redis and Cassandra. It has additional limitations. These are fairly clearly spelled out but maybe weren't highlighted as prominently a few years back. (I say that because I inherited an application that made heavy use of DynamoDB but turned out not to be a great fit for DDB.) - It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty val…

Note that Cassandra has similar limitations with data/throughput, but they aren't enforced or documented (because they depend on your particular setup) and your queries just fail or worse make all queries to the same node in the cluster fail (fun times with large wide rows). The rich data types in Dynamo are quite strange, since they're basically useless for querying I'm not sure why you would use them. Maybe I'm mis…

The rich data types are also useful for partial updates, like adding items to a set, updating some fields in a map etc.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#82

Post by bilalq »

abd12 wrote: post #3

Waves Author here. Happy to answer any questions folks have about the book, about DynamoDB, or about self-publishing. NoSQL modeling is waaay different than relational modeling. I think a lot of NoSQL advice out there is pretty bad, which results in people dismissing the technology altogether. I've been working with DynamoDB for a few years now, and there's no way I'll go back. The book has been available for about a…

Just bought the book. I've been working at AWS and using DynamoDB for years now, but I'm sure there are things I could be doing better. I love that you've dedicated attention to analytics and operations too.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#83

Post by abd12 »

bilalq wrote: post #82
abd12 wrote: post #3

Waves Author here. Happy to answer any questions folks have about the book, about DynamoDB, or about self-publishing. NoSQL modeling is waaay different than relational modeling. I think a lot of NoSQL advice out there is pretty bad, which results in people dismissing the technology altogether. I've been working with DynamoDB for a few years now, and there's no way I'll go back. The book has been available for about a…

Just bought the book. I've been working at AWS and using DynamoDB for years now, but I'm sure there are things I could be doing better. I love that you've dedicated attention to analytics and operations too.

Thank you! I really appreciate it :) Hit me up if you have any questions!

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#84

Post by the_reformation »

danenania wrote: post #76
zemo wrote: post #74

Earlier quoted context omitted.

> I think most apps will need a transaction involving more than 25 rows at some point I ... can't think of a single time I've ever needed this.

A common one is cascading deletes when you delete a user or a ‘project’ or something else that has a lot of stuff associated with it. Those will exceed 25 rows very quickly. Also any kind of bulk update or data import... hell even just initializing a new account can easily require writing more than 25 rows in a moderately complex app.

Still, why the need to be transactional? An eventually consistent delete seems fine here.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#85

Post by maerF0x0 »

whalesalad wrote: post #28
maerF0x0 wrote: post #19

Earlier quoted context omitted.

> makes it difficult to scale up for bursts Can you tell me why the On Demand mode doesnt work for you?

You need to build exponential-backoff logic into your system to handle waiting for Dynamo to warm up. It doesn't happen instantly.

You need that in provisioned in case of overload, too, right?

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#86

Post by danenania »

the_reformation wrote: post #84
danenania wrote: post #76

Earlier quoted context omitted.

A common one is cascading deletes when you delete a user or a ‘project’ or something else that has a lot of stuff associated with it. Those will exceed 25 rows very quickly. Also any kind of bulk update or data import... hell even just initializing a new account can easily require writing more than 25 rows in a moderately complex app.

Still, why the need to be transactional? An eventually consistent delete seems fine here.

Maybe it is, or maybe some of the rows are involved in a security check. Or could cause race conditions if out of sync. Or otherwise need immediate read-after-write consistency.

And eventually consistent isn't the worst case scenario. Being unable to rollback correctly from an error could mean you'll never end up in a consistent state... that's a lot worse than "eventually".

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#87

Post by outworlder »

philipkglass wrote: post #52
arpinum wrote: post #24

I bought the book, I read the book, I've used DynamoDB for awhile. It didn't change my mind. DynamoDB makes tradeoffs in order to run at massive scale, but scale isn't a problem many people need solving when 2TB of RAM fits in a single box. Meanwhile I need to handle eventual consistency, an analytics pipeline, another database for fuzzy search, another geo lookup database, Lambda functions to do aggregations, and a…

Others are comparing DynamoDB to Redis and Cassandra. It has additional limitations. These are fairly clearly spelled out but maybe weren't highlighted as prominently a few years back. (I say that because I inherited an application that made heavy use of DynamoDB but turned out not to be a great fit for DDB.) - It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty val…

> - It provides rich types with some odd limitations: strings, sets, lists, and binaries do not allow empty values.

That is _infuriating_

It's documented, but it is so surprising when you first hit it. Sometimes, empty values have semantics attached to them, I don't want to scrub them out.

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#88

Post by mmbleh »

scarface74 wrote: post #80
mmbleh wrote: post #64

Earlier quoted context omitted.

Differing opinion - I think RDS Proxy is the wrong approach. Adding an additional fixed cost service to enable lambda seems like an indicator of a bad architecture. In this case the better approach would likely be to just use a Fargate container which would have a similar cost and fewer moving parts. By the time you pay a fixed cost for the proxy on top of what you already pay for the RDS server, it'd be a far simple…

I agree completely. We have APIs that are both used by our website and our external customers (we sell our API for our customers to integrate with their websites and mobile apps) and for batch loads for internal use. We deploy our APIs to Fargate for low, predictable latency for our customers and to Lambda [1] which handles scaling up like crazy and scaling down to 0 for internal use but where latency isn’t a concern…

Yup, that's exactly how I recommend clients to write lambdas for API purposes... Such a great balance of getting per request pricing while retaining all existing tooling for building APIs

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#89

Post by jeremyjh »

abd12 wrote: post #38
Nican wrote: post #34

> While your relational database queries slow down as your data grows, DynamoDB keeps on going. It is designed to handle large, complex workloads without melting down. I mean- hand a person a gun, and they might shoot themselves in the foot. While you can make bad queries/workloads for a relational database, you can just as easily make bad workloads for DynamoDB.

My contention is that it's much easier to have an access pattern that won't scale in a relational database than in DynamoDB. DynamoDB basically removes all the things that can prevent you from scaling (JOINs, large aggregations, unbounded queries, fuzzy-search). This is underrated, but it's really helpful. So many times w/ a relational database, I've had to tweak queries or access patterns over time as response times…

So what is the cost of doing a bit of query tuning and de-norming every now and then compared to the development costs imposed by DynamoDB?

Re: The DynamoDB Book: Data Modeling with NoSQL and DynamoDB

#90

Post by scarface74 »

mmbleh wrote: post #88
scarface74 wrote: post #80

Earlier quoted context omitted.

I agree completely. We have APIs that are both used by our website and our external customers (we sell our API for our customers to integrate with their websites and mobile apps) and for batch loads for internal use. We deploy our APIs to Fargate for low, predictable latency for our customers and to Lambda [1] which handles scaling up like crazy and scaling down to 0 for internal use but where latency isn’t a concern…

Yup, that's exactly how I recommend clients to write lambdas for API purposes... Such a great balance of getting per request pricing while retaining all existing tooling for building APIs

For an internal API with one or two endpoints, I‘ll do things the native Lambda way. Your standard frameworks are heavy when all you need to do is respond to one or two events and you can do your own routing, use APIGW and API Key for authorization, etc.

There is also a threshold between “A group of developers will be developing this API and type safety would be nice so let’s use C#” and “I can write and debug this entire 20-50 line thing in the web console in Python, configure it using the GUI, and export a SAM CloudFormation template for our deployment pipeline.”

Post reply on HN