Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing is Hard

chriskiehl.com

Post reply on HN
Print view View original

11–20 of 166 posts

Re: Event Sourcing is Hard

#11

Post by ZenPsycho »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

It seems to be one of those things that you probably don't need. And when you do need it, it becomes obvious that you need it. Specifically, in my research, you really shouldn't use it until it becomes painfully necessary to horizontally scale writes.

Re: Event Sourcing is Hard

#12

Post by ZenPsycho »

the_duke wrote: post #9
linkmotif wrote: post #5

Earlier quoted context omitted.

Event sourcing is REALLY hard to figure out how to do “right.” A lot of getting it right is modeling knowledge/experience, understanding your domain. That said, you can succeed at building your entire arch around it and once you do, it’s glorious. Kafka Streams makes the technical aspects easy once you figure out how to model correctly.

Out of curiosity, how do you deal with consistency guarantees across aggregates? (which is much more relevant when your whole architecture is ES) I realize this is highly domain dependent. Some will be much less affected than others. But it's another drawback not mentioned, because now you start to need sagas/managers that coordinate across services with commit/rollback patterns , conflict resolution., etc.

The architecture obligates you to the "eventually consistent" model. If that's not good enough, you should probably look at other approaches.

Disclaimer: I'm just an avid reader, not an experienced implementor.

Re: Event Sourcing is Hard

#13

Post by 6t6t6t6 »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

EventSourcing is not a Framework, but a concept.

The idea is to store not the current state of your app, but the transitions (events) that derive into the current state.

Think about how git stores your source code as a series of commits.

In theory it is a beautiful idea; in the real world, it is hard to implement.

Re: Event Sourcing is Hard

#14

Post by ralusek »

ZenPsycho wrote: post #10
ralusek wrote: post #8

Here's my take on Event Sourcing: it's not particularly well defined what an "event" is. Did the event happen yet? Did it succeed? Which part? I didn't like having an event ledger say "comment created," where my application is then meant to consume this, handle validation, potentially fail on the db operation, etc. So here is what I do: I basically combine Event Sourcing architecture with CQRS. Whenever a client make…

Having it clear not just in your own head, but your entire team's head what an event is, quite important, especially to keep in mind what happens when you "play back" a log. Do transactional emails get sent out again? Do upstream services record the playback as duplicate events? How are transactions handled? What external state are you unknowingly depending on for that playback to produce the same result?

Ya, I think that is part of the importance of separating out the effects that resulted from the command. If I want to benefit from the "state rewind" capabilities, to bring my DB back to a certain point in time, for example, I can simply play back the DB writes that occurred and perform those updates directly.

Otherwise, I would have to replay the commands, ensure that my application state for processing the commands is exactly the same as it was when it last processed those particular commands, ensure that all undesirable side effects are disabled, etc.

Re: Event Sourcing is Hard

#15

Post by rwilson4 »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

Events are the cornerstone of product analytics. If you want to understand what your users are doing on your platform, and to look for opportunities to improve the user experience, events are a big part of that.

Re: Event Sourcing is Hard

#16

Post by solipsism »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

The past few years there's probably one front-page article a week on Event Sourcing / CQRS.

Re: Event Sourcing is Hard

#17

Post by vikiomega9 »

I think Event Sourcing is neatly defined as an algebraic group and provides a nice abstraction to reason with. Perhaps the problem is not recognizing this notion and not proving the correctness of a system before implementing it?

Re: Event Sourcing is Hard

#18

Post by jokab »

computerex wrote: post #6
AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

It is more widespread in the enterprise software development space and particularly in the Microsoft/.NET ecosystem.

Greg Young was wondering in one of his talks about Event Sourcing why people try to connect CQRS/Event Sourcing with .NET

Re: Event Sourcing is Hard

#19

Post by olalonde »

AndrewKemendo wrote: post #4

The great thing about HN is that it consistently shoves into my face how many, seemingly common, dev tools or frameworks etc... that I've never heard of. Event sourcing isn't anything I've ever heard of, let alone something for which broad marketing promises need debunking. How common is this framework/archecture/product? Google isn't helping me determine how widely used it is.

Things like Redux or Bitcoin are more or less event sourced systems. It's basically the idea of deriving your application state from a series of business events and storing those as a single source of truth. It's very appealing in theory but as the article explains, it's a bit more complicated in practice (e.g. dealing with consistency).

Re: Event Sourcing is Hard

#20

Post by preommr »

Never heard of event sourcing before, but I've used this pattern myself (as another comment mentioned, when you need it, it's a fairly obvious way to do it). Looking it up, I am guessing it's from Martin Fowler's book about patterns and architecture design? And on a related note, would you recommend the book to the average developer?
Post reply on HN