Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing is Hard

chriskiehl.com

Post reply on HN
Print view View original

1–10 of 166 posts

Re: Event Sourcing is Hard

#2

Post by the_duke »

Good summary of the drawbacks of ES.

I think one thing can not be repeated often enough:

Eventsourcing can be an incredibly valuable approach, but almost always only for a very SMALL SUBSET of your system.

Most of the problems with ES materialize from trying to build your whole architecture around it.

I think many developers fall into this trap because theoretically the concept sounds so appealing and elegant (immutable, reproducible, modular, ...)

Re: Event Sourcing is Hard

#3

Post by ZenPsycho »

the_duke wrote: post #2

Good summary of the drawbacks of ES. I think one thing can not be repeated often enough: Eventsourcing can be an incredibly valuable approach, but almost always only for a very SMALL SUBSET of your system. Most of the problems with ES materialize from trying to build your whole architecture around it. I think many developers fall into this trap because theoretically the concept sounds so appealing and elegant (immuta…

I think calling it an architecture is a bit strong. Isn't it more like a pattern that you can use for some of the connective lines in an architecture? Reading through this, it seems a lot of the pitfalls come from neglecting the architecture whilst implementing event sourcing.

Re: Event Sourcing is Hard

#4

Post by AndrewKemendo »

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of.

Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking.

How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

Re: Event Sourcing is Hard

#5

Post by linkmotif »

the_duke wrote: post #2

Good summary of the drawbacks of ES. I think one thing can not be repeated often enough: Eventsourcing can be an incredibly valuable approach, but almost always only for a very SMALL SUBSET of your system. Most of the problems with ES materialize from trying to build your whole architecture around it. I think many developers fall into this trap because theoretically the concept sounds so appealing and elegant (immuta…

Event sourcing is REALLY hard to figure out how to do “right.” A lot of getting it right is modeling knowledge/experience, understanding your domain.

That said, you can succeed at building your entire arch around it and once you do, it’s glorious. Kafka Streams makes the technical aspects easy once you figure out how to model correctly.

Re: Event Sourcing is Hard

#6

Post by computerex »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

It is more widespread in the enterprise software development space and particularly in the Microsoft/.NET ecosystem.

Re: Event Sourcing is Hard

#7

Post by the_duke »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

Eventsourcing is really a lose set of concepts, and each application of it will look very different. That's also why there are almost no useful frameworks here.

There are some good talks on Youtube about it.

The concept is used all over though: Redux (js) is basically a lightweight form of eventsourcing ( with redux-saga being the "process managers" mentioned in the post).

Re: Event Sourcing is Hard

#8

Post by ralusek »

Here's my take on Event Sourcing: it's not particularly well defined what an "event" is. Did the event happen yet? Did it succeed? Which part? I didn't like having an event ledger say "comment created," where my application is then meant to consume this, handle validation, potentially fail on the db operation, etc.

So here is what I do: I basically combine Event Sourcing architecture with CQRS. Whenever a client makes a write request, that "command" gets written to a log. Then when things happen as a consequence of that command, for example, a successful DB write happens, I write another log that references the command.

So I'll have an event that says:

    id: '1234'
    type: 'command'
    data: {type: 'createComment', text: 'Ayy', userId: '1234', postId: '6543'}
And then that can get picked up and processed by the application, which will create another event like:

    id: '2345'
    type: 'dbWrite'
    data: {model: 'comments', data: {text: 'Ayy'}}
    commandId: '1234'
There's a lot of redundancy, and you can instead rely on something like your DB WAL for some of the consequences of the commands, but separating out the commands from the results like this has made Event Sourcing work quite well for me.

Re: Event Sourcing is Hard

#9

Post by the_duke »

linkmotif wrote: post #5
the_duke wrote: post #2

Good summary of the drawbacks of ES. I think one thing can not be repeated often enough: Eventsourcing can be an incredibly valuable approach, but almost always only for a very SMALL SUBSET of your system. Most of the problems with ES materialize from trying to build your whole architecture around it. I think many developers fall into this trap because theoretically the concept sounds so appealing and elegant (immuta…

Event sourcing is REALLY hard to figure out how to do “right.” A lot of getting it right is modeling knowledge/experience, understanding your domain. That said, you can succeed at building your entire arch around it and once you do, it’s glorious. Kafka Streams makes the technical aspects easy once you figure out how to model correctly.

Out of curiosity, how do you deal with consistency guarantees across aggregates? (which is much more relevant when your whole architecture is ES)

I realize this is highly domain dependent. Some will be much less affected than others. But it's another drawback not mentioned, because now you start to need sagas/managers that coordinate across services with commit/rollback patterns , conflict resolution., etc.

Re: Event Sourcing is Hard

#10

Post by ZenPsycho »

ralusek wrote: post #8

Here's my take on Event Sourcing: it's not particularly well defined what an "event" is. Did the event happen yet? Did it succeed? Which part? I didn't like having an event ledger say "comment created," where my application is then meant to consume this, handle validation, potentially fail on the db operation, etc. So here is what I do: I basically combine Event Sourcing architecture with CQRS. Whenever a client make…

Having it clear not just in your own head, but your entire team's head what an event is, quite important, especially to keep in mind what happens when you "play back" a log. Do transactional emails get sent out again? Do upstream services record the playback as duplicate events? How are transactions handled? What external state are you unknowingly depending on for that playback to produce the same result?
Post reply on HN