Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing Is Hard (2019)

chriskiehl.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: Event Sourcing Is Hard (2019)

#61

Post by oskar_dudycz »

nhumrich wrote: post #56
oskar_dudycz wrote: post #41

As this article pops out again, I'd like to the point that although it may have some valid points, those points are not about Event Sourcing. What's expressed in the article is the Event Streaming or Event-Driven approach. So when events are not the source of truth etc. All of the event stores that I know supports strong consistency on appends, optimistic concurrency. Many guarantee global ordering. Some help in idem…

I re-read the article after this comment, and I would have to disagree. The article does never attempt to explain "what" recent sourcing means to them, so it's hard to know for sure. However, they do mention populating state from an event log which contains meaningless events, so I have to assume they _are_ talking about event sourcing. What leads you to believe they are taking about just streaming, and not sourcing?

Could you provide an exact quote? I haven't been able to find any usage of "populate". I found:

`the raw event stream subscription setup kills the ability to locally reason about the boundaries of a service.`

`you have to talk to the people who will be consuming the events you produce to ensure that the events include enough data for the consuming system to make a decision`

`wire a fleet of services together via an event stream`

etc.

Plus the drawing that's at the beginning.

This clearly states that the author doesn't understand what's Event Sourcing and tells about Event Streaming. Event Streaming means:

1. Producer pushes/produces/publishes an event to the queue. In this sense, an event stream is a pipe where you put an event at the end and read it on the other side.

2. Then, you have a set of subscribers that can react to those events. The reaction may be triggering the next step of the workflow. It might also be updating the read model state. That happens with eventual consistency. You also cannot guarantee atomicity or optimistic concurrency, as tools like Kafka doesn't support it. As they were not originally built for. They let you effectively publish and subscribe to the messages.

3. Because of that, if you want to have the write model materialised from an event (or even "populated"). Then you don't have any guarantee if it's stale or not. You need to fight with out of order messages and idempotency. Thus all of those issues that are described in the article.

I'm sorry to say, but the author used the wrong tool for the wrong job. If this article be titled "Event Sourcing is hard if you're confusing it with Event Streaming" then it'd be an excellent article. But, in its current shape, it's just misleading and making the wrong point, repeating just common misunderstandings.

You can read more in:

- https://domaincentric.net/blog/eventstoredb-vs-kafka

- https://www.eventstore.com/event-sourcing

Re: Event Sourcing Is Hard (2019)

#62

Post by oskar_dudycz »

kreetx wrote: post #46
oskar_dudycz wrote: post #41

As this article pops out again, I'd like to the point that although it may have some valid points, those points are not about Event Sourcing. What's expressed in the article is the Event Streaming or Event-Driven approach. So when events are not the source of truth etc. All of the event stores that I know supports strong consistency on appends, optimistic concurrency. Many guarantee global ordering. Some help in idem…

BTW, does there happen to be a discord channel for this topic? ;)

There is a Slack channel: https://join.slack.com/t/ddd-cqrs-es/shared_invite/zt-m3vf3a...

Re: Event Sourcing Is Hard (2019)

#63

Post by Kototama »

the-alt-one wrote: post #23

The big issue, to me, with concepts such as event sourcing is that they are fundamentally presented as software engineering concepts. You get a nice picture showcasing the overall structure and a story or two of how succesful this is, but we never discuss computer science matters. I want to know things such as "what safety properties does this system afford me?", "how is it abstractly modelled?", "what global invaria…

How does that differ from other architecture patterns?

Re: Event Sourcing Is Hard (2019)

#64

Post by dkarl »

minimeme wrote: post #48
akiselev wrote: post #19

I've worked with several event sourcing systems and was even seduced into implementing one out of sheer hubris once. These problems are ever present in every ES project I've had the misfortune of coming into contact with. It doesn't even mention the worst part that comes afterwards, when you realize after all of that pain that it is only used by a single person in the company to generate a noncritical report comparin…

Event sourcing can be a very powerful pattern if used correctly. You don't need to combine ES with Eventual Consistency. ES can be implemented in a totally synchronous manner and it works very elegantly capturing nicely all the busines events you need. You get a very detailed audit for free and you don't loose importan business data. You can travel back in time, construct different views of data (projections) etc. Mo…

I agree. Audit and history functionality have been the motivating features for me in building systems that are "ES-lite," where the event stream is not a general-purpose API and is only consumed by code from the same project.

Sometimes the ability for an engineer to fetch the history of a business object out of a datastore and explain what it means checks an "audit trail" requirements box. Sometimes showing the history of a business process in a UI, with back-in-time functionality, is a game changing feature. If so, using event sourcing internally to the service is a great way to ensure that these features use the same source of truth as other functionality.

Where you get into trouble is when you realize that event sourcing will let you distribute the business logic related to a single domain object to a bunch of different codebases. A devil on your shoulder will use prima facie sound engineering logic tell you it is the right thing to do. The functionality relates to different features, different contexts, different operational domains, so keeping it in the same service starts to feel a bit monolithic.

But in practice you probably don't have enough people and enough organizational complexity to justify separating it. What happens is, a product manager will design an enhancement to one feature, they'll work out the changes needed to keep the product experience consistent, and the work will get assigned to one engineer because the changes are reasonable to get done in one or two sprints. Then the engineer finds out they have to make changes in four different systems and test six other systems for forwards compatibility. Oops. Now you have nanoservices-level problems, but only microservices-level capabilities.

If you aren't tempted down that path, you'll be fine.

Re: Event Sourcing Is Hard (2019)

#65

Post by oskar_dudycz »

dm3 wrote: post #60
nhumrich wrote: post #56

Earlier quoted context omitted.

I re-read the article after this comment, and I would have to disagree. The article does never attempt to explain "what" recent sourcing means to them, so it's hard to know for sure. However, they do mention populating state from an event log which contains meaningless events, so I have to assume they _are_ talking about event sourcing. What leads you to believe they are taking about just streaming, and not sourcing?

They were probably also sourcing their state from the events. However most of their problems come from sharing the events between the modules/services which is not a part of Event Sourcing.

I think that they were just building the stale read models, and used them as the write model, which created the whole confusion. Regarding the sharing events between module, it's one of the most common and the most dangerous mistakes. It's a leaking abstraction that will eventually create a distributed monolith. It has only downsides of monolith and microservices, without the upsides. I wrote longer on the topic of internal and external events, and how to model them: https://event-driven.io/en/events_should_be_as_small_as_poss...

Re: Event Sourcing Is Hard (2019)

#66

Post by picardo »

adamauckland wrote: post #54

Event Sourcing is absolutely brilliant if you want to build an offline first/distributed system which will become eventually consistent. A good example would be a tree inspection application where jobs can be pushed out to mobile inspectors who might not have phone signal but can still collect the data. Once synchronised, views are simply additive. More data can be added to cases simply by adding events. I would abso…

> A good example would be a tree inspection application where jobs can be pushed out to mobile inspectors who might not have phone signal but can still collect the data. Once synchronised, views are simply additive. More data can be added to cases simply by adding events.

This isn't a great use case for Event Souring either. An inspector must be in front of the tree to inspect it, right? And they can see each other inspecting the tree.... Why would several inspectors inspect the same tree?

Assuming they inspect the same tree at different times, you can sync offline data at the row level, and let the last update win, provided that they're entering the same data.

Re: Event Sourcing Is Hard (2019)

#67

Post by bob1029 »

DrBazza wrote: post #49

I've worked on a few event sourcing systems, and the only ones that have been truly successful are the ones where the system was written from scratch. Trying to retrofit systems never seemed to quite work as expected. Also, the "from scratch" systems, seemed conceptually easier to not only understand, but extend, because you were forced to write code in an idiomatic and consistent way. Everything was an event (albeit…

I am currently looking at a new iteration of our product that uses an event log as the principal data store. I agree that if we start from "zero" and don't even provide the option to build things the wrong way it would go smoothly. "everything is an event" is very powerful if its not just some add-on policy.

For us, the entire working set would be able to live in memory. Larger objects would be left as pointers to the actual data in the log. Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates.

Re: Event Sourcing Is Hard (2019)

#68

Post by DrBazza »

bob1029 wrote: post #67
DrBazza wrote: post #49

I've worked on a few event sourcing systems, and the only ones that have been truly successful are the ones where the system was written from scratch. Trying to retrofit systems never seemed to quite work as expected. Also, the "from scratch" systems, seemed conceptually easier to not only understand, but extend, because you were forced to write code in an idiomatic and consistent way. Everything was an event (albeit…

I am currently looking at a new iteration of our product that uses an event log as the principal data store. I agree that if we start from "zero" and don't even provide the option to build things the wrong way it would go smoothly. "everything is an event" is very powerful if its not just some add-on policy. For us, the entire working set would be able to live in memory. Larger objects would be left as pointers to th…

> Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates.

Exactly this. We took hourly snapshots (for a trading system), and could restart the system in a second or so.

Another nice side effect of an event driven system is that you start to think about what other data you can put in there. And then you realise you don't necessarily need a database any more, which makes the architecture of the system simpler.

There are so many benefits working with a data-driven system. Once data is in the log/ledger/whatever you call it, it's immutable, which leads to parallelism if you want it. Ensuring the system is idempotent also means you get (easier) failover as well.

Re: Event Sourcing Is Hard (2019)

#69

Post by the-alt-one »

Kototama wrote: post #63
the-alt-one wrote: post #23

The big issue, to me, with concepts such as event sourcing is that they are fundamentally presented as software engineering concepts. You get a nice picture showcasing the overall structure and a story or two of how succesful this is, but we never discuss computer science matters. I want to know things such as "what safety properties does this system afford me?", "how is it abstractly modelled?", "what global invaria…

How does that differ from other architecture patterns?

This isn't unique to event sourcing, I definitely took the opportunity to soapbox.

Re: Event Sourcing Is Hard (2019)

#70

Post by larve »

hvgk wrote: post #51
akiselev wrote: post #19

I've worked with several event sourcing systems and was even seduced into implementing one out of sheer hubris once. These problems are ever present in every ES project I've had the misfortune of coming into contact with. It doesn't even mention the worst part that comes afterwards, when you realize after all of that pain that it is only used by a single person in the company to generate a noncritical report comparin…

This is a common problem I see across many things. We had a guy who spent two weeks writing a report script that collated data uploaded into S3 and wrote out another data lump into S3 after some post processing then sent this data lump to a guy via email. This entire thing had a bunch of lambda layers to pull in the pipeline, a build pipeline in jenkins, terraform to deploy it. The python script itself was about 200…

I wouldn't use the overengineering of a single script influence your view of the technology you named (S3 data laking, lambda pipelines, build in jenkins, terraform to deploy). Those are well understood concepts and technologies to do reporting. The problem here is that these tools were not in use at the company. Once you have 50 scripts run by different people on different schedules in different excel version, having everything go through S3 (immutable, cheap), ingested by lambdas (scalable, well versioned, monitored), built and deployed in jenkins using terraform (vs say, aws command line scripts) significantly reduces the complexity of the problem.
Post reply on HN