Live data from Hacker News

Why Are People into Event Sourcing?

adaptechsolutions.net

41–50 of 90 posts

Re: Why Are People into Event Sourcing?

#41
post #31
post #13

Earlier quoted context omitted.

Agree with the potential for complexity. Here's how we've dealt with this (on a so far / so good basis): Didn't seem necessary to go asynchronous and beef up on heavy infrastructure, so we went with simple in-thread, in memory message buses to start with. I think a lot of the perceived complexity of building event sourced systems comes because people start with the heavy plumbing instead of going YAGNI in order to ge…

This seems solid advice. I guess my main questions are: * If you are in-memory and in-process, why bother with the events in the first place? (Simpler put, why not go with the simpler process based solutions?) * If you are not testing distributed, how do you know you will be able to distribute? In particular, there is a very large chance that you will have the same difficulty in replacing an in-memory/process solutio…

I don't really do eventsourcing for technical reasons. To me it's about creating an environment for project delivery where teams can succeed because they own more of their stack, from an org chart & organizational perspective.

As far as code is concerned, I see that as coming down to managing coupling & cohesion in such a way that the various pieces can be designed, built, supported, deployed and enhanced by the same team, with minimal need to wait for, coordinate with or be on the same page with any other team.

I think event sourcing & CQRS are great for enabling that because they result in the lowest form of coupling: Commands are fire and forget. You don't care who subscribes to your events. You don't care who publishes events you subscribe to. You never query anything you don't own in order to get information you need for your business logic. While the system is still small, we seem to get these advantages just fine when we do in-memory and in-process. Once package management, performance, units of deployment, etc. become an issue, we build out to add infrastructure, as needed. Because the pieces are pretty standalone within the code, it's not all that complicated to do, compared to "traditional" CRUD systems.

Not testing distributed: It's possible to deduce a lot with testing locally if it's simple to understand what's coupled to what and if good shared contracts for commands and events are in place, but ultimately the proof is of course in the pudding. When the time comes, there is a big advantage: The system is already fully operational and testing comes down to "does it still work?" If one were to do the plumbing before the business functionality and tests that in isolation ... does that really amount to knowing that it'll still be OK once it actually runs what it needs running ...? For starters, there is a danger of over-engineering because the required performance characteristics of the end products are more difficult to asses if there is no end product yet.

Re: Why Are People into Event Sourcing?

#43
post #36
post #35

Earlier quoted context omitted.

Slightly shooting from the hip here (as I'm still learning). Low-ish volume- design your system such that data flows [at the relevant crucial points] through a single actor to ensure proper concurrency. High volume- trickier but I think same idea in principle. First thought that comes to mind here is the new GenStage stuff in Elixir.

I'm not sure what you are suggesting. There is no getting around "at most/at least once." You can shift the posts, some, but at some point that is your choice. This is a good read on the problem: http://bravenewgeek.com/you-cannot-have-exactly-once-deliver...

"Exactly once delivery" is not the same as "One message at a time". Akka Actors process one message at a time. Akka does no provide exactly once delivery. It default defaults to "at most once".

Re: Why Are People into Event Sourcing?

#44

Very curious: if you have multiple datastores, how do you ensure they are consistent? If you scale sideways, how do you ensure nothing gets lost if there's a partition? Etc?

Embrace eventual consistency. A good deal of collaborative domains (things involving human decisions) are naturally eventually consistent. Meat computers appear to be particularly good at resolving conflicts and compensating.

Re: Why Are People into Event Sourcing?

#45
post #3

As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard. The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understoo…

> Moving everything to be an event completely throws this out the window.

This is either an antipattern or unrelated to event-sourcing depending on how you read it. "Event sourced" means that state within a transaction boundary is built ONLY through events which are regarded as persistent and immutable; evented is the term I'll use for state which is built when events happen, when those events may or may not be recorded, and may happen before or after state is changed and are independent of the state change.

If you meant "evented" then I would say that there are lots of message-based systems that aren't failing hard, and a lot that are, which says to me that there are patterns some of those systems are using to manage the nature of async, evented development that others aren't.

If you mean "event sourcing" then the application of event-sourced data is neither an application architecture nor appropriate for all areas of your application[1,2]. If you were trying to apply event-sourcing in this way its not surprising you ran into problems with it.

> Until then, my main advice is to make sure you have solved your system with a naive solution before you move on.

It is important to really know and understand the problem domain you are applying ES to. Having a strategy to upgrade streams to new versions of your domain model is a good idea if you're applying ES to a business without a well-understood domain. However, it is very, very difficult to back into an ES implementation from a "naive" solution, which I'm reading as "CRUD".

[1] https://www.youtube.com/watch?v=LDW0QWie21s [2] https://www.infoq.com/news/2016/04/event-sourcing-anti-patte...

Re: Why Are People into Event Sourcing?

#46
post #43
post #36

Earlier quoted context omitted.

I'm not sure what you are suggesting. There is no getting around "at most/at least once." You can shift the posts, some, but at some point that is your choice. This is a good read on the problem: http://bravenewgeek.com/you-cannot-have-exactly-once-deliver...

"Exactly once delivery" is not the same as "One message at a time". Akka Actors process one message at a time. Akka does no provide exactly once delivery. It default defaults to "at most once".

So, then to go back to the original.

You may not have to do deal with locks at a local level. You absolutely have to deal with locks at a system level.

Re: Why Are People into Event Sourcing?

#47

I've tried working out how to move to an event sourcing system, but I always struggle with locking behavior. Do you just have to invent your own locking mechanisms on top of event sourcing?

The stream is the consistency (locking) boundary. Your first step it to get your model aligned with such boundaries. For example, your amazon shopping basket is independent of my basket. Then you chose your concurrency model - append to a stream with an expected version (pessimistic) or just append anyway (no expected version). Your amazon basket may be the later, your amazon payment and shipping checkout may be the former.

Locking across streams is an anti-pattern / smell. It can be done (as can anything) but it usually points to a modelling problem. Example: cancelling an amazon order is a _request_ that is in a race with the fulfillment system (boundary); it may or may not be successful.

Re: Why Are People into Event Sourcing?

#48
post #7

Earlier quoted context omitted.

We find that a simple in-memory synchronous message bus + event logging to files goes a long way. See e.g. https://github.com/robertreppel/hist for an in-memory bus + file system (and DynamoDB ...) helloworld which isn't .net. Scaling that up by adding asynchronicity and more ambitious plumbing when needed seems reasonably straightforward. For something more out-of-the-box, see https://geteventstore.com/ . It has cli…

Are you using DynamoDB Streams at all? I've been toying the idea of using DynamoDB as an event store and having other services listen to a table's stream, allowing them to update caches/views (the read-side of CQRS), report analytics, perform asynchronous tasks, etc.

You could quite nicely use AWS Lambda for the materialized views, I think.

Re: Why Are People into Event Sourcing?

#49

There is a lot that could be done to make event sourcing easier to work with... Imagine tooling that allowed an event stream to be used to create state for testing modules, crudlike helpers to allow crud-familiar developers to think that way at first, and workflows based on snapshots, rewind, etc. I think a model that used events that correlated to graph deltas rather than crud deltas would be the cat's ass, and many…

> Imagine tooling that allowed an event stream to be used to create state for testing modules, crudlike helpers to allow crud-familiar developers to think that way at first, and workflows based on snapshots, rewind, etc.

i know where you're going with this, and i honestly believe its a terrible idea (not to be discouraging or rude—just experienced.)

if your event streams contain mostly CRUD (possibly ANY) then you're most likely applying it incorrectly. Its not just a version history of your data. The event type itself is data, which provides context and semantics over and above the notion of writes and deletes. If you're falling back to crud events all you're doing is creating a lot more work for yourself and deriving almost no benefit from the use of ES—in that case, you should just use CRUD and the ORM of your choice.

Re: Why Are People into Event Sourcing?

#50
post #12
post #3

As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard. The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understoo…

I always hoped that materialized views would rise to the occasion and we could have the best of both worlds. Events for writes and current data for most reads.

Just don't do eventual consistency if the domain doesn't allow for it and if performance is OK. From a functional perspective there is no need for it in CQRS/ES systems.

However ... if going with eventual consistency, it's essential that the write side is consistent. The read models - not so much. This is because every aggregate (... if Domain Driven Design terminology is your thing) represents a transactional boundary within which state needs to be consistent in order for the business rules to work correctly. Read models are more forgiving.

Post reply on HN