Live data from Hacker News

Event Sourcing is Hard

chriskiehl.com

101–110 of 166 posts

Re: Event Sourcing is Hard

#101

Interesting- Sounds to me like you were reinventing a database - PostgreSQL has taken over 30 years, has 400 contributers and 1.1 Million LOC, no surprise to me, it was a bit tricky! PG has a rock solid ACID enabling transaction log - the Event Source - you can access this easily via logical decoding functions. You can easily replicate this data into tables, if you need to keep it, you can also add system and applica…

just wanted to say that I don't like your comment. too shallow, wrong direction - something along those lines

Re: Event Sourcing is Hard

#102
post #97

I've worked in a couple of ES based trading system (matching engines and algo trading both) and it was definitely the way to go, especially on the equity side were you can restart or snapshot the system every day. In my startup, I've designed the core transactional system and it's serving us well for 2 years now. It comes with its own challenges and you need more senior devs than your average crud system hence you us…

I think this is the kind of pragmatism that is needed when taking about event sourcing. It is a _very_ advanced architectural pattern. There is no easy 'Event Sourcing made simple' way to use it.

I think one of the reasons for this is that the systems people often describe when they talk event sourcing are actually _three_ different, but interrelated, architectural patterns:

- event sourcing (build your models based on immutable facts) - event driven (side affects triggered by messages, often delivered by queues) - workflow / state machine

It takes a long time to get these concepts straight. In our case once it was untangled, our framework can quite clearly demonstrate how they relate: messages update the state in a workflow engine, this triggers side effects, the results are captured as facts and used to build the model, repeat.

This worked for our use case, in which our transactions have a strong workflow, it may not work in other cases.

Finally the one point I'd certainly reinforce from the article is: _don't reach directly into the event stream_. This causes huge amounts of coupling. Instead, we ended up using bounded contexts to define our systems, and then treat key events as our API. It sounds counter to some of the ideals of event sourcing, but it is absolutely needed once you grow past that toy phase.

Re: Event Sourcing is Hard

#103

I agree with the author completely. I worked on a fairly large system using event sourcing, it was a never-ending nightmare. Maybe with better tooling someday it will be usable, but not now. Events are pretty much a database commit log. This is extremely space innefficient to keep around. And not nearly use useful as you might think. Re-runs need to happen pretty often as you change how events are handled. Even in ou…

I agree that it's hard, however doable and pays benefits if you know what you're doing. I worked on 3 successful implementations for finance sector and we could replay a few million messages per second. Have a look at how we achieved that in LMAX: https://martinfowler.com/articles/lmax.html Sorry to say it, but clearly you must have been doing something wrong or employing event sourcing where it does not belong.

> LMAX's in-memory structures are persistent across input events, so if there is an error it's important to not leave that memory in an inconsistent state. However there's no automated rollback facility. As a consequence the LMAX team puts a lot of attention into ensuring the input events are fully valid before doing any mutation of the in-memory persistent state. They have found that testing is a key tool in flushing out these kinds of problems before going into production.

I'm sorry, but this is saying "catch your bugs before they reach production" which just isn't feasible on non-critical software development (i.e., most software development). The important part that is left out here is: what happens when one such errors slips in? How do you deal with it after the fact?

That being said, your system is impressive and I loved being able to read about it. Please keep up the good work and specially sharing your findings! :)

Re: Event Sourcing is Hard

#104
post #42

This is question more than a comment, as I have only casual knowledge of event sourcing... """ You wouldn't let two separate services reach directly into each other's data storage when not event sourcing – you'd pump them through a layer of abstraction to avoid breaking every consumer of your service when it needs to change its data """ Isn't the event itself precisely that layer of abstraction? That is, you're not p…

> You're publishing an event which is a thin slice or crafted combination of details that ultimately reside in that store

The event stream is the canonical store.

Re: Event Sourcing is Hard

#105

Datomic is, at its core, an event sourced datastore, and it works really well. I don’t think event sourcing is something that should be solved in application space — you wouldn’t write a database from scratch for your products yet implementing an in-house event sourcing system is for some reason more acceptable. All of these stories of failure are stories of teams bandaiding event sourcing on top of other databases n…

I'm using Datomic in a production environment and the ergonomics of development so far have been extremely nice. Would definitely recommend.

Re: Event Sourcing is Hard

#106

Datomic is, at its core, an event sourced datastore, and it works really well. I don’t think event sourcing is something that should be solved in application space — you wouldn’t write a database from scratch for your products yet implementing an in-house event sourcing system is for some reason more acceptable. All of these stories of failure are stories of teams bandaiding event sourcing on top of other databases n…

Datomic: Event Sourcing without the hassle https://vvvvalvalval.github.io/posts/2018-11-12-datomic-even...

Re: Event Sourcing is Hard

#108

It is worrying that a central figure to Event Sourcing & CQRS like Greg Young reduces the "framework" to a function, a pattern match & a left fold. Linked in the article https://youtu.be/LDW0QWie21s?t=1926

[deleted]

Re: Event Sourcing is Hard

#109
post #52

It is hard to separate DDD, Event Sourcing/CQRS - they seem to all be joined up concepts promoted by a small circle of people. I worked on a blockbuster project financed by a local billionaire in a Gulf State where the entire shebang was “mandated” by the CTO and a well-known Scala consultancy. Event sourcing, CQRS, with DDD to define architecture. Let me describe one simple issue that was almost intractably complex…

I might be misinterpreting your example, but wouldn't you have both the auth service and the preferences service listening for a 'new user' event (which would contain all the details needed by both systems), and both if them acting on it? Of course it would get more complex to handle error conditions, e.g what do you do if there is validation on the preferences that fails...

Re: Event Sourcing is Hard

#110
post #98

Earlier quoted context omitted.

> Save all the events and you can always get to the latest state This isn't actually true and good event sourcing guides will point out why this isn't true. Event based systems are naturally racey, on a rerun of a stream of events the order in which the events is processed may change and you might therefore get a different result than the first time. For example, if you have 1 item remaining in inventory but two peop…

There should be 1 point in the system that said "user x has ordered item a". That then becomes an event. The command from user x and y to order item a might be a race condition to see which command gets processed first, but once it has been turned into an event it's immutable and without race conditions. You should never replay commands.

Never done event sourcing architecture, but i guess this kind of subtelty is what makes a system a nightmare or a breeze to work with.

Which makes me wonder if there are some kind of exercise book to practice « good architectural design » , just like with algorithms.

Post reply on HN