The Reactive Monolith – How to Move from CRUD to Event Sourcing

Maybe I'm a bit slow but I'm not seeing how any of their line-of-business operations are improved through this architecture.

This seemed like a great rundown on why event sourcing is complicated: https://medium.com/@hugo.oliveira.rocha/what-they-dont-tell-...

Can this be mitigated by ensuring that historical/legacy events are all transformed to more modern equivalents? Similar in theory to https://stripe.com/blog/api-versioning ? Then you at least have a chain of testable transformations and a reduced cardinality for the actual interactions that reflects the cardinality of all the ways your business users actually use your product today.

Of course, this can be lossy, but is it any more lossy than the non-event-sourced CRUD system which is essentially transforming historical events into "overwrite these fields" events that can only be used for their original purpose?

To me, the reason not to use event sourcing is more that the tooling and developer experience is just really poor - and so decisions like the above, while valuable to think through directly, rapidly become overwhelming. Visualizations of system state are only as good as the code you bring yourself. I'm hopeful that people building tools on top of things like https://materialize.com will start to change this though!

Event sourcing could be a great architecture for general purpose applications - if we had good frameworks and specialized event stores built for this purpose.

Some features that would be needed:

* Automatic , schema based event and aggregate versioning

* A combined event and aggregate data store with ACID guarantees, schema validation and indexes

* event migrations (so you don't have to carry around old event versions forever)

* Event stream migrations that allow re-writing streams for purging data, gdpr cleanup, etc

* Copy on write based stream clones that can be used for testing with production data

As it stands, such a solution is nowhere to be seen. So you have to cobble together a working system with lots of custom work , which often ends in a complete mess.

I second this wholeheartedly because this happened to me.

I'm the kind of person who learns by my making my own mistakes (as opposed to learning from others', which is what smart people do)—and boy, did I learn a lot from the mistake of trying to build a general purpose application with event sourcing.

Theoretically, event sourcing is the way all applications should be built. I was first taken by this methodology by Martin Kleppmann's talk "Turning the Database Inside Out" (2014)[0]. Looking back, I think I had always subconsciously felt like there was something wrong with mutating state in place. You lose information. You lose accountability. After all, "accountants don't use erasers," and Martin's talk validated many of my obsessive-compulsive insecurities about data. As Charity Majors points out on Twitter [1]:

> ...while storing raw events may be "expensive" compared to any single optimization, all optimizations can be derived from raw events. The reverse is impossible.

And that's true. The table-stream duality [2][3] means that if you store events, you can later project those events into the exact perfect table for any use-case. But if you transform those events into a projection and store that projection, well, whatever table you store is data you have. You wind up having just an aggregate.

Unfortunately, the technical costs you incur when you attempt to store events instead of tabular data are tremendous. The problems and questions that arise are basically endless. Tons of things you wouldn't even think about with a relational database become complicated design decisions [4]. If there were tooling and frameworks that made this easy, it would be one thing. But, there are not!

Kafka Streams is very cool. If you already have a running business with well-developed models and a large team, event sourcing with Kafka Streams could possibly go quite well. But if you need to get an MVP off the ground, it's just not what you need. What you need is PostgreSQL, perhaps with something like Hasura or postgraphile in front of it. What you need is a lightning-fast way to iterate and flesh out your idea. If that's you, and you're reading this, and you're smart: learn from me—event sourcing: don't do it!

[0] https://www.youtube.com/watch?v=fU9hR3kiOK0

[1] https://twitter.com/mipsytipsy/status/1115537408705957888

[2] https://docs.confluent.io/platform/current/streams/concepts....

[3] https://news.ycombinator.com/item?id=23207123

[4] https://softwareengineering.stackexchange.com/questions/3801...