What they don’t tell you about event sourcing

It does change the scaling math, but don't automatically assume that every ES+CQRS system is intended for thousand-writes-per-second, terabytes-per-day kind of scales. If the system stays at ten-writes-per-second, megabytes-per-day, a read store (beyond in-memory projections), a queue or an indexer are not necessary.

As for asynchronous error handling, and delays between reads/writes : why would that be a consequence of ES+CQRS ? It would be a consequence of implementing ES+CQRS with asynchronous or eventually consistent behaviour...

TLDR: orm’s + large transactions resulted in a lot of unexpected complexity.

At some point you get really big transactions because one write triggers five process managers which all trigger more writes and so on and so on. Performance was not a problem but I was surprised by the complexity of these big transactions in combination with an orm.

I dont have a concrete example but over the course of two years we have encountered multiple bugs that took days to solve. Theres one of these bugs that we fixed without identifying the root cause until this day.

Because the events coalesce into kafka, you can use a mechanism like logstash to spew this log data into eg elastic search and then use that for your queries. Or write to the db here (in process) or there (cqrs style.) There are different architectural approach used, and they all yield good results, of very high reliability in processing, if slightly "eventual."

It works well. We do use rdbms too, but a lot of the time we avoid reading from it after initialization (or after encountering an exception causing an actor to restart). Depends on the data though. Some places I read on every command because I know the risk front is small and it's easier for a jr to get in there and understand what's happening. A good rule is that a single process should own that data so you don't have to worry much about consistency related concerns. In microservices good practices, each service should really own its own data completely but I believe that it's fair enough to say a bounded context owns its own data (even if in a shared rdbms) if building at a smaller scale.

In terms of code, I'm usually building with some variation of onion architecture these days, where all context is build on the outside (eg receiving a request, getting info from state or db), and pure domain logic 100% effect free exists on the inside that receives all of the data from the outside and turns out some commands or events in response. This makes the important logic very testable, and easy to reason about. Core domain logic has no effects - not even logging - but only generates events/commands in response to commands and events. You never mock that stuff - it's the star of the show. Nor do you ever have to. The generated commands and events are later "applied" by the outer layer, logging, shipping them off to kafka, and/or writing some stuff to a database.

That's where I'm at today. I am working in realtime systems so it's a good fit for the approaches. I'm finding the software is turning out very well. We're working fairly quickly, the software is very encapsulated to a bounded context, the domain logic is clear in the core, easy to test and read, and change. Or even rewrite if needed.

I see this issue raised quiet often. If consistency is paramount you can make commands on certain aggregates be synchronous all the way to updating the read model. There's nothing that says you MUST have a queue in-between the event log for an aggregate and the logic that updates the read model. Use common sense.

Typically shines the most when pinpointing parts of the system that benefit from it, identifying a specific bounded context in DDD terms, but never on a whole system.

I feel like Greg Young has taken great pains to make this clear. This should be taken for granted when attempting CQRS.

Also your events will be based on a SomethingCreated or SomethingUpdated which has no business value at all. If the events are being designing like this then it is clear you’re not using DDD at all and you’re better of without event sourcing. Finally, depending on the requirements on how the synchronous the UI and the flow of the task is the eventual consistency can, and most of the times will, have a klinky feel to it and deliver a poor user experience.

If the read and write model are being updated asynchronously from the UI you're gonna have to adopt an optimistic caching scheme on the client. This is why GraphQL subscriptions are pretty much boilerplate for any client I build against a CQRS service. The Apollo client seems to handle this rather well.

Converting data between two different schemas while continuing the operation of the system is a challenge when that system is expected to be always available. Due to the very nature of software development new requirements are bound to appear that will affect the schema of your events that is inevitable.

I hereby give you permission to use the Strategy Pattern. Problem solved.

The events can’t be too small, neither too large they have to be just right. Having the instinct to get it right requires an extensive knowledge of the system, business and consumer applications, it’s very easy to choose the wrong design.

Greg Young and others have talked quite a bit about how to bound aggregates.

However the events in a event store are immutable and can’t be deleted, to undo an action means sending the command with the opposite action.

This is why bookkeeping systems have the idea of "journal entries". I haven't implemented one for an event sourced system but I can see how this might work.

Overall great post. Really enjoyed that the author took the time to walk us through all of these issues. Most are non-trivial.