CQRS: "Command Query Responsibility Segregation" Fowler's post is predictably excellent: https://www.martinfowler.com/bliki/CQRS.html
This is already cited and linked in the article (along with Fowler's article on event sourcing).
What they don’t tell you about event sourcing
21–30 of 81 posts
Re: What they don’t tell you about event sourcing
#22Regarding eventual consistency, a CQRS\ES system can also be synchronous, or partially. You could have listeners for events that need to supply a strongly consistent model and others events that feed parts of the system that don't need strong consistency. "However the events in a event store are immutable and can’t be deleted, to undo an action means sending the command with the opposite action" Well they don't have…
> can also be synchronous
Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity.
> Well they don't have to be immutable. I don't see why you can't update/migrate events.
The events are your source of truth. When you "migrate" your source of truth, you're in somewhat dangerous waters (and if you're using CQRS for scale, a 1% failure to migrate data might be millions of records). You also now have the issue that your source of truth is being migrated, and probably can't accept writes (or you need to emit writes in both new format and old format for whenever your change over happens)
Re: What they don’t tell you about event sourcing
#23It's really important to manage versioned schema for events and defining rules around evolution of schemas, hopefully with tooling support like via Avro's Schema Registry or Protobuf's forward and backwards compatibility guarantees.
Re: What they don’t tell you about event sourcing
#24Regarding eventual consistency, a CQRS\ES system can also be synchronous, or partially. You could have listeners for events that need to supply a strongly consistent model and others events that feed parts of the system that don't need strong consistency. "However the events in a event store are immutable and can’t be deleted, to undo an action means sending the command with the opposite action" Well they don't have…
> can also be synchronous Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. > Well they don't have to be immutable. I don't see why you can't update/migrate events. The events are your source of truth. When you "migrate" your source of truth, you're in somewhat dangerous waters (and if you're using CQRS for scale, a 1% failure to migrate data migh…
>> can also be synchronous
> Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity.
Even without asynchronous read/write, there are still benefits worth the (arguably, small) additional complexity. For instance, the ability to add new functionality without having to migrate existing data is amazing.
Re: What they don’t tell you about event sourcing
#25Earlier quoted context omitted.
> can also be synchronous Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. > Well they don't have to be immutable. I don't see why you can't update/migrate events. The events are your source of truth. When you "migrate" your source of truth, you're in somewhat dangerous waters (and if you're using CQRS for scale, a 1% failure to migrate data migh…
>> can also be synchronous > Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. Even without asynchronous read/write, there are still benefits worth the (arguably, small) additional complexity. For instance, the ability to add new functionality without having to migrate existing data is amazing.
You still have migrations, but they exist on the read store, which means they can be done semi-transparently to clients (pause writes and let them queue up, migrate existing read store to new instance, point read calls and indexers to new store, resume writes).
CQRS adds a lot of complexity. Sometimes it's absolutely worth it, especially if you've already invested in the expertise and tooling to support it. It drastically changes the scaling math, both on the low end (at the very least you need a write store, a read store, a queue, an indexer and an api) and on the high end (you can scale any part of the system as needed in relative isolation). You add in timing issues, rollbacks, asynchronous error handling, and delays between reads/writes.
Re: What they don’t tell you about event sourcing
#26CQRS & Event Sourcing are good ideas, but putting yourself ON PURPOSE with eventual consistency is nuts.
Until your are in the facebook-big-data scaling challenging, you don't need the madness of lose ACID. You can get terabytes of data easily on a single server, and even partition by company, customer or similar that still allow to keep domain-level consistence.
Now, I put the events after my normal CRUD operations (ie: I emit CRUD + Save to event in a single transaction). Is super easy of operate and keep coding familiar and predictable
----
I think the ideal EE database engine must be like:
Have a log Table, and a index/subtables for each validation (ie: to check uniques, aggregates, counts, etc.)
So, if a have a customer related events, I have:
- Index on: code, name
- SubTable: code, name, isactive
all the other fields are not need for validation so are recovered from the log.
In ACID:
- POST Command
- Validate data with the index,
- Save to log
- Emit blocking events (events that need to be at the same time after the save)
- Commit
Eventual:
- Emit lazy events (events that not need ACID, like to fill external sources)
Re: What they don’t tell you about event sourcing
#27Still like the idea but it is more complex to do it right and keep it right.
Would only do this with a good team and a real problem where this would be useful.
Rdbm goes very far
Re: What they don’t tell you about event sourcing
#28Sounds like OLTP transactions and aggregated state only with hip names.
Re: What they don’t tell you about event sourcing
#29I've worked with event sourced systems quite a bit in production and at scale, and have written a book on Akka and another one on related topics. While I have been an advocate of the approach, My experiences are guiding me away from implementing Event Sourcing in many use cases (especially where the entities are long lived). While CQRS is more complex, I'm more likely to implement CQRS without event sourcing, where a…
Elixir is fantastic, especially in these use cases because it allows for stateful in-memory representations of an entity with less development burdens than elsewhere.
I find it useful to have a GenServer for complex entities like state machines modeling business processes. I'm fine with the simple entities using the database schema as the state definition. However with the complex entities I still find I run into the classic ORM problem where the database structure doesn't perfectly fit the domain's pure business logic representation.
Re: What they don’t tell you about event sourcing
#30Earlier quoted context omitted.
>> can also be synchronous > Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. Even without asynchronous read/write, there are still benefits worth the (arguably, small) additional complexity. For instance, the ability to add new functionality without having to migrate existing data is amazing.
You still have migrations, but they exist on the read store, which means they can be done semi-transparently to clients (pause writes and let them queue up, migrate existing read store to new instance, point read calls and indexers to new store, resume writes). CQRS adds a lot of complexity. Sometimes it's absolutely worth it, especially if you've already invested in the expertise and tooling to support it. It drasti…
I don't think you necessarily have to have all this in a useful CQRS system.
E.g., here's a real-world example of a general pattern in which CQRS is pretty simple and useful:
A walking/running app which tracks distance, time, and other relevant information over the course of a workout.
It collects a series of events like "location changed at time X", "user started workout", "user paused workout" etc., over the course of the workout period (actually starting before the user officially starts the workout), and converts these to time, distance and other stats.
This fits CQRS really well since the input is inherently a series of events and the output is information gleaned from processing those events.
You get a full CQRS system by simply fully logging the events.
The advantage is that you can go back and reprocess the sequence if you want to glean new/different information from the sequence. E.g., in the walking/running app you could, after the fact and at the user's discretion, detect and fix the case where the user forgets to start or stop a workout. Or recalc distance in the case where you detect a bug or misapplication of your smoothing algorithm, etc. Or draw a pace graph or whatever.
In all these cases you can process the events synchronously.
I put this all in terms of a workout app, but there is a general pattern of an event-driven session-based activity or process, where you may want/need to derive new information from the events and the cost is to log the events (in a high-fidelity form so they could be fully reproduced.
Whether or not you need to use queues and distributed data stores is an independent decision.