I would be interested in knowing how the reactors are handling side-effect which should never be replayed. Is there some well established pattern for doing this?
Combining event sourcing and stateful systems
11–20 of 41 posts
Re: Combining event sourcing and stateful systems
#12I would be interested in knowing how the reactors are handling side-effect which should never be replayed. Is there some well established pattern for doing this?
Reactors can keep their own state including their current position in the event stream. When a replay is initiated it ignores events older than it's current "head."
Re: Combining event sourcing and stateful systems
#13You should take a look at Microsoft's Durable Functions which pairs event sourcing + (optional) actor model + serverless. It's some pretty neat tech. I tried doing something similar to this several years ago, and here's a few issues I ran into: 1. Pub/sub in Event Sourcing is a bad idea. It's really hard to get right. (what to do if sub happens after pub due to scaling issues/infrastructure, etc?) Instead it's better…
On [1] - You're correct that Pub/Sub is difficult to get right, but it can confer a bunch of benefits which make it worthwhile to struggle with. RabbitMQ + Replay-ability (the details of which will differ based on your design) + good data model design is usually a safe bet here.
Re: Combining event sourcing and stateful systems
#14I think to truly be an event-driven architecture you need to go a step or two further and be data-driven.
In other words, the appropriate way to describe your system would not be (subscribable) relationships between a set of components that describe your presumptive view of a division of responsibilities. (This is the non-event driven way of doing things, but with the arrows reversed.)
Instead, you track external input types, put them into a particular stream of events, transform those events to database updates or more events, etc. Your entire system is this graph of event streams and transformations.
These streams may cut across what you thought were the different responsibilities, and you will have either saved yourself headaches or removed a fatal flaw in your design.
If you're thinking about doing work in this area, don't just reverse the arrows in your component design!
Re: Combining event sourcing and stateful systems
#15Earlier quoted context omitted.
It took several hours of individual research, watching talks, reading blog posts; and took several pair-programming sessions of several hours over the span of four weeks to come up with a solution we liked. We informed our client that this was a new area for us and that we didn't have hands-on experience with, but that we believed it would be beneficial to spend time to explore it, as it would be an elegant solution…
How do you approach estimating effort for this sort of thing? I find it awkward enough in Scrum to guess up front how many days of effort research will take and commit to delivering a plan or design by the end. If you have clients and aren't strictly bound by someone else's framework, they still want some rough idea how long research will take. Especially if the client is footing the bill. If the research is un-bille…
Reaction Commerce behind an Airframe static site up quick.
Have an http server between the two, and log the requests.
Users don't update products or orders directly. The critical data that needs to be logged runs on a private network. Something with a lot of validation updates that.
Don't delete services when creating new features. Keep them running.
Avoid a central source of truth. Instead log discrepancies. Then your procedure is "Don't do this" along with "do this".
A lot of microservices are built starting as low as you can go.
I prefer to install proven working software and close it to modification. Have a bunch of them running together instead of making something new.
Re: Combining event sourcing and stateful systems
#16You should take a look at Microsoft's Durable Functions which pairs event sourcing + (optional) actor model + serverless. It's some pretty neat tech. I tried doing something similar to this several years ago, and here's a few issues I ran into: 1. Pub/sub in Event Sourcing is a bad idea. It's really hard to get right. (what to do if sub happens after pub due to scaling issues/infrastructure, etc?) Instead it's better…
Is (2) motivated by using aggregates which do not commute or by trying to do distributed modifications on a single value?
One common technique if you have commutative aggregates is to have each writer just write to their own spot and then do a range query on read to re-join. If your aggregates don’t commute this, of course, doesn’t work and you’re stuck in “single threaded” land. I do remember reading a paper that avoided this, but i can’t remember the implementation / trade offs (If I can find the paper I’ll post it here)
Re: Combining event sourcing and stateful systems
#17Re: Combining event sourcing and stateful systems
#18Earlier quoted context omitted.
It took several hours of individual research, watching talks, reading blog posts; and took several pair-programming sessions of several hours over the span of four weeks to come up with a solution we liked. We informed our client that this was a new area for us and that we didn't have hands-on experience with, but that we believed it would be beneficial to spend time to explore it, as it would be an elegant solution…
How do you approach estimating effort for this sort of thing? I find it awkward enough in Scrum to guess up front how many days of effort research will take and commit to delivering a plan or design by the end. If you have clients and aren't strictly bound by someone else's framework, they still want some rough idea how long research will take. Especially if the client is footing the bill. If the research is un-bille…
If you are perceived as just a cog delivering software patches, you've already lost. If you're presenting business proposals and designs, you've already provided research, analysis and business cases. Often this is alot of extra unrewarded work. Though, if a vendor has proven themselves already, they're in position to ask for billable time for more different types of roles. This upfront work is an investment in own business and can be used to attract other clients, but sure, most of it goes down the toilet unless a good outlet is found for all that creative work. A bigger consultancy can have many people cooperating on such work in order to add that extra value to hiring companies.
Re: Combining event sourcing and stateful systems
#19I'm working with an event sourced system and we did some mistakes in the process of the design, so some areas that didn't need event sourcing do have it.
The biggest downside has been the UI: events are not real time and these objects are just CRUD stuff, so the user wants to see that you saved what he has just written. You might not have this information yet, so you need to mitigate it, for example updating the UI through sockets (lot of additional work).
On the upside, we are acquiring a lot more insights in what business processes bring value and are meaningful, versus what I call "just configuration".
We figured out quite a few rules of thumb over time that are helpful though.
One thing I noticed over time is that on average there is no need for "created" and "updated" event, usually there is one meaningful business event that would encompass both (not always the case), e. g. "product listed", or something along those lines. This not only saves lines, but some code reacting to this event has a reduced interaction surface (less bugs and coupling), as well as being more expressive.
If you might be interested, we chat a lot about event sourcing in the Eventide Slack channel: https://eventide-project.org/#community-section
Re: Combining event sourcing and stateful systems
#20 Events + state = state machine
(Technically, I was let go. Friday I went to head of HR and said, "I think I'm gonna quit." Monday morning I was laid off. * shrug *)