The Reactive Monolith – How to Move from CRUD to Event Sourcing

The problem is that the CRUD is only recording "What" change/mutation was done, but not "Why" by "Whom", and "When" it was done. With tailing WAL / CDC you can also capture the "When".

This can be partially mitigated by adding audit fields like:

  Why (reason for change):

  created_bc ("because") / created_reason   (a person opened an a/c, a new baby born, or a record was migrated from another datastore)
  updated_bc ("because") / updated_reason   (fixed typo in the address, or moved to new address)
  deleted_bc ("because") / deleted_reason   (closed an a/c, an end-user died, GDPR request, soft-deleted, removed by moderator, etc.)

  By Whom (i.e. user_id, support person, decision maker, etc.):

  created_by
  updated_by
  deleted_by

  When (event time) - with CDC you can capture all the intermediate updated_at events:

  created_at
  updated_at
  deleted_at

But if you have the luxury to do consistent CRUD with a relational/graph, ACID database, then a temporal data model might give you much more leverage.

Essentially you think of your records as bookkeeping entries and go from there. You retain the ability to do ad-hoc relational queries, while not having to build core database functionality yourself.

The industry is moving chaotically and sometimes getting stuck in a local minima for a decade. The innovation follows diffusion model, and it slowly builds momentum until the tipping point is reached.

So I do agree, that CQRS/ES might be the next informal industry standard, but we do need tooling, best practices, use cases and success stories.

Maybe we need a special programming language, a DSL, or a framework for it, or a special DBMS.

Another point: it's not that easy to build a correct, scalable and maintainable moderate-complexity CRUD app. Anything beyond a simple blog engine or TODO list tutorial can quickly become a mess. The tutorials are omitting error handling, and that where the juniors are learning from.

Last anecdotal datapoint: when building a moderate-complexity CRUD app, and trying to handle all edge cases/error handling/correctness, I realized that I'm building a poor man's CQRS, so I might do the real thing as well.

if you use a timeseries databases like kdb+/q, TimescaleDB, InfluxDB, QuestDB, etc., you don't even need to order by the insertion timestamp.

Unlike RDBMS/SQL where rows/tuples are unordered, in timeseries databases they're usually ordered by the insertion time, more similar to dataframes than to SQL tables.

--

[1] Martin Kleppmann — Event Sourcing and Stream Processing at Scale

https://www.youtube.com/watch?v=avi-TZI9t2I