Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What they don’t tell you about event sourcing

medium.com

Post reply on HN
Print view View original

31–40 of 81 posts

Re: What they don’t tell you about event sourcing

#31

Post by marcosdumay »

raarts wrote: post #18
ninjakeyboard wrote: post #14

I've worked with event sourced systems quite a bit in production and at scale, and have written a book on Akka and another one on related topics. While I have been an advocate of the approach, My experiences are guiding me away from implementing Event Sourcing in many use cases (especially where the entities are long lived). While CQRS is more complex, I'm more likely to implement CQRS without event sourcing, where a…

I've re-read your post a couple of times, but still find it confusing. It reads like you're an early-adopter, and got burnt multiple times by design flaws, and people not being familiar with it. Suppose ES/CQRS is designed well, and a well-understood mechanism, would you still move away from it? For which scenarios? I can imagine exactly-once delivery not being a problem in all scenarios. And what is the problem with…

The GP sounds exactly like the "you don't need microservices for everything, you can have well built monoliths too", "not everything must be a single page app", or "not every datacenter should be replaced with a cloud one".

ES specifically has a large list of drawbacks, so not everybody should use it. It would be great if people could tell about how there are many changes of gray between "CRUD-only we forget everything that is not current" and ES, but in a hush to make a point people act like those are the only known possibilities.

Re: What they don’t tell you about event sourcing

#32

Post by vorpalhex »

jmull wrote: post #30
vorpalhex wrote: post #25

Earlier quoted context omitted.

You still have migrations, but they exist on the read store, which means they can be done semi-transparently to clients (pause writes and let them queue up, migrate existing read store to new instance, point read calls and indexers to new store, resume writes). CQRS adds a lot of complexity. Sometimes it's absolutely worth it, especially if you've already invested in the expertise and tooling to support it. It drasti…

I don't think you necessarily have to have all this in a useful CQRS system. E.g., here's a real-world example of a general pattern in which CQRS is pretty simple and useful: A walking/running app which tracks distance, time, and other relevant information over the course of a workout. It collects a series of events like "location changed at time X", "user started workout", "user paused workout" etc., over the course…

That's not "Command Query Responsibility Segregation" (CQRS). That's modeling your data as a time series - which is a totally valid and perfectly useful model in many cases, but has nothing to do with the architectural pattern known as CQRS.

Martin Fowler gives the following simple definition of CQRS:

> At its heart is the notion that you can use a different model to update information than the model you use to read information.

CQRS takes data (events, time series, plain old records, whatever), stores it into a write store which acts as a singular source of truth, and queues that data to be stored in a (usually eventually consistent) read store as a projection - not simply duplicating the write store, but building a read record meant to be consumed directly by some client - with potentially several projections, one for each set of client needs.

Re: What they don’t tell you about event sourcing

#33

Post by voiceofunreason »

dmitriid wrote: post #11

Is it just me or the article never presented a solution to the posited problem: ”Since each entity is represented by a stream of events there is no way to reliable query the data. I have yet to meet an application that doesn’t require some sort of querying.” The solution is said to be CQRS, and nothing in the article shows how you solve that. If you unwind the unnecessarily winded florid poetic waxing, it all comes d…

"dump it to a database, and query that"

Yes.

In some cases, "dump" is a fold/reduce, and your database is just an in memory data structure, and depending on how much latency is permitted by your service level objectives you might cache the data structure as opposed to regenerating it every time.

There's no magic.

The pattern is analogous to what you would do if your book of record were an RDBMS, and you had to run graph queries. "Dumping" the data into a graph database and running the query there would be a tempting solution, no?

Re: What they don’t tell you about event sourcing

#34

Post by jwhitlark »

zihotki wrote: post #20
7sigma wrote: post #8

Regarding eventual consistency, a CQRS\ES system can also be synchronous, or partially. You could have listeners for events that need to supply a strongly consistent model and others events that feed parts of the system that don't need strong consistency. "However the events in a event store are immutable and can’t be deleted, to undo an action means sending the command with the opposite action" Well they don't have…

I also share same opinion based on my experience. Events can be modified and deleted but that must be an exceptional situation (GDPR and other compliances, etc.). But even if it's exceptional you have to provide a clear and easy way to do so and that increases complexity of the solution by a lot. Another thing is strongly consistent models, there may be valid requirements in some problem areas to have a strongly cons…

Encryption + lose the key policies seem to satisfy the GDPR. So you don't have to actually delete an event, you can just give up your ability to read its payload.

Re: What they don’t tell you about event sourcing

#35

Post by spitfire »

lscharen wrote: post #16

The last section on Operational Flexibility and the inability to change the event history raises a very good point. Like most of the issues, the solution requires experience to know when you are at the Goldilocks point (Just Right). This specific issue has a lot in common with managing database migrations in django or any other migration system. The ideal situation is to create migrations that can always be rolled ba…

Basically implement double entry accounting.

I think ideally you want something like Rich Hickey speaks about when he speaks of Datomic. An append only database. You can see what the previous values for that row were, along with schema changes.

Re: What they don’t tell you about event sourcing

#36

Post by ninjakeyboard »

raarts wrote: post #18
ninjakeyboard wrote: post #14

I've worked with event sourced systems quite a bit in production and at scale, and have written a book on Akka and another one on related topics. While I have been an advocate of the approach, My experiences are guiding me away from implementing Event Sourcing in many use cases (especially where the entities are long lived). While CQRS is more complex, I'm more likely to implement CQRS without event sourcing, where a…

I've re-read your post a couple of times, but still find it confusing. It reads like you're an early-adopter, and got burnt multiple times by design flaws, and people not being familiar with it. Suppose ES/CQRS is designed well, and a well-understood mechanism, would you still move away from it? For which scenarios? I can imagine exactly-once delivery not being a problem in all scenarios. And what is the problem with…

I'm only saying it isn't the only approach, even if you're keeping the source of truth in memory. It's the most prescriptive approach for sure so is the one people tend to go for first but there be dragons in managing especially long lived journals. Migrating the journal over time, figuring out how to truncate it (especially if there is an initial creation command/event that describes the rest of the life, or periodic updates to the structure - you have to keep all of that data forever potentially to know that it's going to wind up being correct.)

And yes long recovery times are an issue unless you want to keep every entity in memory forever and ever, or you can tolerate extremely slow turnaround times. There are knobs and dials here that are subtleties that people won't think about until after they launch.

If you have a hammer everything looks like a nail kind of thing. There are other ways to handle problems that are only marginally different in how they persist and recover state, yet are far more usable for certain use cases. I'm extremely skeptical when I see someone gung-ho for event sourcing if they've never used it though. I tend to look at the problem very hard to see what else we could do for persistence while still maintaining the "source of truth" in memory.

Re: What they don’t tell you about event sourcing

#37

Post by beiller »

LandR wrote: post #5
ulcica wrote: post #2

They didn't even tell me what event sourcing is.

As I understant it: Consider you have a database where you store the account balance. If you want to update the account balance you might update the row for that customer, e.g. tblAccounts ------------ | AccountHolderId | AccountBalance | update tblAccounts set AccountBalance = @NewAccountBalance; In an EventSource database instead you wouldn't update the AccountBalance column. You would store something like: Account…

Account balance is a bad example, because it cannot be solved using event sourcing. A key to event sourcing is eventual consistency. The account balance cannot be eventually consistent, otherwise it will allow double-spending. It has to be immediately consistent.

Re: What they don’t tell you about event sourcing

#39

Post by UK-Al05 »

beiller wrote: post #37
LandR wrote: post #5

Earlier quoted context omitted.

As I understant it: Consider you have a database where you store the account balance. If you want to update the account balance you might update the row for that customer, e.g. tblAccounts ------------ | AccountHolderId | AccountBalance | update tblAccounts set AccountBalance = @NewAccountBalance; In an EventSource database instead you wouldn't update the AccountBalance column. You would store something like: Account…

Account balance is a bad example, because it cannot be solved using event sourcing. A key to event sourcing is eventual consistency. The account balance cannot be eventually consistent, otherwise it will allow double-spending. It has to be immediately consistent.

Not if the event sourcing aggregate rehydrates on the request to spend, and sends failure or success back.

Re: What they don’t tell you about event sourcing

#40

Post by theptip »

I don't see much discussion of event-sourcing simply using a SQL database (i.e. skipping the CQRS part). This would allow you to keep your CP (strongly-consistent) semantics.

While this clearly wouldn't work in high-volume cases (i.e. where you _actually_ need CQRS), it seems like this would be the simplest option for many systems. I see a lot of articles advocating for immediately jumping into CQRS, which seems like a big increase in architectural complexity.

Does anyone have opinions/experience on this approach?

Post reply on HN