Rules for Data Modeling with DynamoDB

This is great advice, wish I had it before I started working with DynamoDB. Hot keys can be a particularly nasty landmind.

"Normalization was built for a world with very different assumptions. In the data centers of the 1980s, storage was at a premium and compute was relatively cheap."

But forget to do normalisation and you will be paying 5 figures a month on your AWS RDS server.

"Storage is cheap as can be, while compute is at a premium."

This person fundamentally does not understand databases. Compute has almost nothing to do with the data layer - or at least, if your DB is maxing on CPU, then something is wrong like a missing index. And for storage, its not like you are just keeping old movies on your old hard disk - you are actively accessing that data.

It would be more correct to say: Disk storage is cheap, but SDRAM cache is x1000 more expensive.

The main issue with databases is IO and the more data you have to read, process and keep in cache, the slower your database becomes. Relational or non-relation still follows these rules of physics.

> You need to manage data integrity in your application

This is just another way of saying you need to implement your own system for managing consistency. Dynamo offers transactions now, but they don’t offer actual serialization. Your transaction will simply fail if it runs into any contention. You might think that’s ok, just retry. But because you’ve chosen to sacrifice modelling your data, this will happen a lot. If you want to use aggregates in your Dynamo, you have to update an aggregate field every time you update your data, which means you create single points of contention for your failure prone transactions to run into all over your app. There are ways around this, but they all come at the cost of further sacrificing consistency guarantees. The main issue with that being that once you’ve achieved an inconsistent state, it’s incredibly difficult to even know it’s happened, let alone to fix it.

Then you run into the issue of actually implementing your data access. With a denormalized data set, implementing all your data access comes at the expense of increasing interface complexity. Your schema ends up having objects, which contain arrays of objects, which contain arrays... and you have to design all of your interfaces around keys that belong to the top level parent object.

The relational model wasn’t designed to optimize one type of performance over another. It was designed to optimize operating on a relational dataset, regardless of the implementation details of the underlying management system. Trying to cram a relational set into a NoSQL DB unavoidably comes at the expense of some very serious compromises, with the primary benefit being that the DB itself is easier to administer. It’s not as simple as cost of storage vs cost of compute. NoSQL DBs like dynamo (actually especially dynamo) are great technology, with many perfectly valid use cases. But RDBMS is not one of them, and everybody I’ve seen attempt to use it as an RDBMS eventually regrets it.

I can set up a DynamoDB based serverless API that scales automatically in a minute. I wouldn't call this an effort.

It's just that you know Postgres and I know DynamoDB. So go with what you know :)

In my experience (using Dynamo for 4-5 use cases while Postgres for many more) a properly configured HA Postgres with RDS costs way more than Dynamo for the same workload. There is only a big catch, data modeling is not as straightforward with Dynamo. You cannot change it up so easily than with Postgres and your access patterns must cover every single use case. If you can do that, model your data and the access pattern fits Dynamo great, if not you are going to have a hard time.