Why Are People into Event Sourcing?
adaptechsolutions.net
Why Are People into Event Sourcing?
1–10 of 90 posts
Re: Why Are People into Event Sourcing?
#2I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet.
We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's another microservice that lets you query past events for statistics.
Re: Why Are People into Event Sourcing?
#3As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard.
The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understood and actually quite visible in the code.
Moving everything to be an event completely throws this out the window. You can take a naive view, where you pretend from one event to the next is safe to happen. However, to start building up the system to cope when this is not the case starts to build a complicated system. In areas that are decidedly not related to your business domain. (Well, for most of us.)
Maybe some day there will be a system that helps with this. Until then, my main advice is to make sure you have solved your system with a naive solution before you move on.
Re: Why Are People into Event Sourcing?
#4Event sourcing isn't nearly as common knowledge among new programmers as the CRUD-one-row-per-entity pattern, and it really should be. I liken it to introducing version control for your data; when immutable updates are your canonical source, no matter how much the system behind them changes, or the business requirements change, and no matter how many teams are deriving different things from them in parallel, they can all work off of the same data and "merge" their efforts together.
The one downside is that shifting your business logic to read-time means that you need to have very efficient ways of accessing and memoizing derived data. For some applications, this can be as simple as having the correct database indices over your WhateverUpdates tables, fetching all updates into memory and merging on each request. For others, you'll need to have a real-time stream processing pipeline to preemptively get your derived data into the right shape into a cache. And those are more moving parts than your typical monolith app, but the
One benefit to actually using event sourcing with a stream processing system is that, in many cases, it can be the most effective way to scale both traffic capacity and organizational bandwidth, much in the same way that individually scalable microservices can (and fully compatible with that approach!). Martin Kleppman at Confluent (a LinkedIn spinoff creating and consulting on stream processing systems) writes some great and highly-approachable articles about this. Highly recommended reading.
http://www.confluent.io/blog/making-sense-of-stream-processi...
http://www.confluent.io/blog/turning-the-database-inside-out...
Re: Why Are People into Event Sourcing?
#5FYI in case the author reads this, since this seems to be intended as an intro for people who aren't already familiar with this stuff: I didn't see "CQRS" defined anywhere in this article or in the two or three links I followed from it; they all begin with an assumption that you know the acronym, and delve straight into details. It might be good to define some terms in the front matter (unless I've misunderstood the target audience).
Re: Why Are People into Event Sourcing?
#6As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard. The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understoo…
Do you have specific examples of where things break hard?
We are currently using ES end to end for a distributed application including a wearable device, iOS app and scala backend, and up to now, the things that broke hard in the system are the naive/non ES parts.
For information, on the server-side, we are currently experimenting with GetEventStore (https://geteventstore.com/), which seems to be working well for us.
Re: Why Are People into Event Sourcing?
#7I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet. We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's ano…
We find that a simple in-memory synchronous message bus + event logging to files goes a long way. See e.g. https://github.com/robertreppel/hist for an in-memory bus + file system (and DynamoDB ...) helloworld which isn't .net.
Scaling that up by adding asynchronicity and more ambitious plumbing when needed seems reasonably straightforward. For something more out-of-the-box, see https://geteventstore.com/ . It has clients in a variety of languages. Comes with a nice HTTP API too.
I wouldn't normally read the entire event stream; usually, only the state of a particular object (aggregate, in Domain Driven Design speak) is of interest, E.g. the customer with id 12345. Events contain the aggregate ID, so the query to whatever event store you use would be "give me all events with aggregate ID 12345".
Re: Why Are People into Event Sourcing?
#8I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet. We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's ano…
>Just didn't seem to be a "standard" answer yet.
This article was extremely helpful to me for understanding some solutions in this space.
http://www.confluent.io/blog/turning-the-database-inside-out...
Re: Why Are People into Event Sourcing?
#9Axon Framework http://www.axonframework.org is a great place to start if you're into Java and want to get a feeling for how event sourcing works.
There's also a great presentation by the developer, Allard Buijze, at https://www.youtube.com/watch?v=s2zH7BsqtAk.
Re: Why Are People into Event Sourcing?
#10FYI in case the author reads this, since this seems to be intended as an intro for people who aren't already familiar with this stuff: I didn't see "CQRS" defined anywhere in this article or in the two or three links I followed from it; they all begin with an assumption that you know the acronym, and delve straight into details. It might be good to define some terms in the front matter (unless I've misunderstood the…
Always a problem with techie - acronymania. :) Thanks, noted. I'll do an edit.