Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Are People into Event Sourcing?

adaptechsolutions.net

Post reply on HN
Print view View original

11–20 of 90 posts

Re: Why Are People into Event Sourcing?

#11

Post by samidalouche »

impostervt wrote: post #2

I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet. We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's ano…

You basically consider the event log as a big collection,and you "fold over" the events in order to incrementally build your state/projection, the same way you would do with finite collections in a Functional language (scala, haskell, ...).

GetEventStore documentation has some examples of how you can create projections (https://geteventstore.com/blog/20130212/projections-1-theory...), which you can use as inspiration to build your own projections.

Re: Why Are People into Event Sourcing?

#12

Post by hinkley »

taeric wrote: post #3

As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard. The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understoo…

I always hoped that materialized views would rise to the occasion and we could have the best of both worlds.

Events for writes and current data for most reads.

Re: Why Are People into Event Sourcing?

#13

Post by rreppel »

taeric wrote: post #3

As someone that has fallen for the "event sourcing" promise before, the article does a decent job explaining the promise. Not sure if it will be the next article, but the actual task of delivering on this work is where things break. Hard. The vast majority of the things you will ever program are pretty much guaranteed from one statement to the next. Hard boundaries, where things can fail, are often decently understoo…

Agree with the potential for complexity. Here's how we've dealt with this (on a so far / so good basis): Didn't seem necessary to go asynchronous and beef up on heavy infrastructure, so we went with simple in-thread, in memory message buses to start with. I think a lot of the perceived complexity of building event sourced systems comes because people start with the heavy plumbing instead of going YAGNI in order to get the domain model implemented. Easier to beef things up as needed once everything works.

I became disillusioned with doing the naive solution, for two reasons:

Found it to be impossible (from a time/project mgmt perspective) to ever replace it with the "non-naive" one, so it turns into the usual mess because CRUD doesn't work well as you load more functionality on it over time.

Secondly ... it's thinking machines. From a business perspective, does it still make sense to hand code glorified rolodexes without behaviour? Maybe Excel does the trick. Seeing it as a red flag if someone asks me to build dumb data entry forms in 2016.

Therefore, I always start eventsourced theses days. YMMV.

Re: Why Are People into Event Sourcing?

#14

Post by UK-AL »

impostervt wrote: post #2

I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet. We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's ano…

Either you have fast queries and indexes, or you have a microservice that monitors for certain events, and keeps up to date state in a cache.

Re: Why Are People into Event Sourcing?

#15

Post by burnout1540 »

rreppel wrote: post #7
impostervt wrote: post #2

I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet. We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's ano…

We find that a simple in-memory synchronous message bus + event logging to files goes a long way. See e.g. https://github.com/robertreppel/hist for an in-memory bus + file system (and DynamoDB ...) helloworld which isn't .net. Scaling that up by adding asynchronicity and more ambitious plumbing when needed seems reasonably straightforward. For something more out-of-the-box, see https://geteventstore.com/ . It has cli…

Are you using DynamoDB Streams at all? I've been toying the idea of using DynamoDB as an event store and having other services listen to a table's stream, allowing them to update caches/views (the read-side of CQRS), report analytics, perform asynchronous tasks, etc.

Re: Why Are People into Event Sourcing?

#16

Post by rreppel »

burnout1540 wrote: post #15
rreppel wrote: post #7

Earlier quoted context omitted.

We find that a simple in-memory synchronous message bus + event logging to files goes a long way. See e.g. https://github.com/robertreppel/hist for an in-memory bus + file system (and DynamoDB ...) helloworld which isn't .net. Scaling that up by adding asynchronicity and more ambitious plumbing when needed seems reasonably straightforward. For something more out-of-the-box, see https://geteventstore.com/ . It has cli…

Are you using DynamoDB Streams at all? I've been toying the idea of using DynamoDB as an event store and having other services listen to a table's stream, allowing them to update caches/views (the read-side of CQRS), report analytics, perform asynchronous tasks, etc.

No, haven't tried them yet. Does look interesting for it, though.

Re: Why Are People into Event Sourcing?

#17

Post by barrkel »

Architecting around events has several ramifications.

For building up a picture of the world, it's pretty good. It's very nice to be able to replay a log of events and recreate a view of the way things are expected to be; if there's a bug in your code, you can fix it and repeat the replay to get back into a good state (with caveats, sometimes later actions creating events may be dependent on an invalid intermediate state). Whereas mutating updates erase history, perhaps with some ad-hoc logging on the side that is more often than not worthless for machine consumption.

For decoupled related action, it's not too bad. If you have some subsystem that needs to twiddle some bits or trigger an action when it sees an event go by, it just needs to plug into the event stream, appropriately filtered.

For coordinated action OTOH, e.g. a high-level application business-logic algorithm, you need to start thinking in terms of explicit state machines and, in the worst case, COMEFROM-oriented programming[1]. Depending on how the events are represented, published and subscribed to, navigating control flow involves repeated whole-repo text searching.

It's best if your application logic is not very complicated and inherently suitable to loose coupling, IMO.

[1] https://en.wikipedia.org/wiki/COMEFROM

Re: Why Are People into Event Sourcing?

#19

Post by jessaustin »

kazagistar wrote: post #18

I've tried working out how to move to an event sourcing system, but I always struggle with locking behavior. Do you just have to invent your own locking mechanisms on top of event sourcing?

ISTM event sourcing actually avoids many locking problems, since it's essentially "write-only". Of course every event write should be atomic, but that seems easier than making updates atomic?

Re: Why Are People into Event Sourcing?

#20

Post by GundersenM »

Having been part of a project to rewrite a monolith e-commerce site into an event-sourced, domain driven, CQRS system, let me tell you in which situation that is not possible: when you already have data. Remember that in a DDD, ES, CQRS system, the event store is the single source of truth. If you already have data in a relational database, then the existing data is the source of truth. You can't have two sources of truth, that completely defeats the purpose. So it's not actually possible to migrate to an event sourced system, you can only create one from scratch, with no existing data.
Post reply on HN