Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing in Go: From Zero to Production

skoredin.pro

Post reply on HN
Print view View original

41–50 of 55 posts

Re: Event Sourcing in Go: From Zero to Production

#41

Post by cedws »

LVB wrote: post #39
cedws wrote: post #38

Stuffing data into JSONB columns always makes me feel uncomfortable. Not necessarily for performance/efficiency reasons. You also loose the strong schema that SQL gives you, you don't get to use constraints. You might as well use Mongo, no? How can you be sure that the data stuffed into JSONB fits a particular schema, and that future changes are backwards compatible with rows added long ago?

JSONB can have constraints. I think with an extension you can do full JSON Schema validation, too.

Yes, I watched a video[0] about using CHECKs and pg_jsonschema to do this in the past. However this only checks for conformance at insert/update time. As time goes on you'll inevitably need to evolve your structures but you won't be able to have any assurances to whether past data conforms to the new structure.

The way this article suggests using JSONB would also be problematic because you're stuffing potentially varying structures into one column. You could technically create one massive jsonschema that uses oneOf to validate that the event conforms to one of your structures, but I think it would be horrible for performance.

[0]: https://www.youtube.com/watch?v=F6X60ln2VNc

Re: Event Sourcing in Go: From Zero to Production

#43

Post by coredev_ »

There are a lot of voices against event sourcing in the comments. I'd just want to balance things a bit. For a mature domain (like when you rebuild an existing system), event sourcing can work really well and make so much sense. But yes, dicipline is a must, as well is thinking things through before you implement a new event.

Currently working on a DDDd event sourced system with CQRS and really enjoy it.

Re: Event Sourcing in Go: From Zero to Production

#44

Post by simonw »

anthonylevine wrote: post #34
simonw wrote: post #29

Earlier quoted context omitted.

I've not run an event sourcing system in production myself. This thread appears to have stories from several people who have though, and have credible criticisms: https://news.ycombinator.com/item?id=45962656#46014546 https://news.ycombinator.com/item?id=45962656#46013851 https://news.ycombinator.com/item?id=45962656#46014050 What's your response to the common theme that event sourcing systems are difficult to mainta…

I think having constantly changing product requirements would certainly make it difficult, but that makes all development more difficult. In fact, I think most complexity I create or encounter is in response to trying to future-proof stuff I know will change. I'm in healthcare. And it changes CONSTANTLY. Like, enormous, foundation changes yearly. But that doesn't mean there aren't portions of that domain that could b…

Thanks, this is a great comment. Love the observation that event sourcing only makes sense for parts of a system.

Could be that some of the bad experiences we hear about are from people applying it to fields like content management (I've been tempted to try it there) or applying it to whole systems rather than individual parts.

Re: Event Sourcing in Go: From Zero to Production

#45

Post by mrkeen »

simonw wrote: post #24
quapster wrote: post #19

[flagged]

This comment just made it finally click for me why event sourcing sounds so good on paper but rarely seems to work out for real-world projects: it expects a level of correct-design-up-front which isn't realistic for most teams.

> it expects a level of correct-design-up-front which isn't realistic for most teams.

The opposite is true.

A non-ES system is an ES system where you are so sure about being correct-up-front that you perform your reduce/fold step when any new input arrives, and throw away the input.

It's like not keeping your receipts around for tax time (because they might get crinkled or hard to read, or someone might want to change them).

Re: Event Sourcing in Go: From Zero to Production

#46

Post by mrkeen »

simonw wrote: post #29
anthonylevine wrote: post #27

Earlier quoted context omitted.

No you can't. You can blame the endless amount of people that jump in these threads with hot takes about technologies they neither understand or have experience with. How many event sourced systems have you built? If the answer is 0, I'd have a real hard time understanding how you can even make that judgement. In fact, half of this thread can't even be bothered to look up the definition of CQRS, so the idea that "Sto…

I've not run an event sourcing system in production myself. This thread appears to have stories from several people who have though, and have credible criticisms: https://news.ycombinator.com/item?id=45962656#46014546 https://news.ycombinator.com/item?id=45962656#46013851 https://news.ycombinator.com/item?id=45962656#46014050 What's your response to the common theme that event sourcing systems are difficult to mainta…

> What's your response to the common theme that event sourcing systems are difficult to maintain in the face of constantly changing product requirements?

I've been on an ES team at my current job, and switched to a CRUD monolith.

And to be blunt, the CRUD guys just don't know that they're wrong - not their opinion about ES - but that the data itself is wrong. Their system has evaluated 2+2=5, and with no way to see the 2s, what conclusion can they draw other than 5 is the correct state?

I have been slipping some ES back into the codebase. It's inefficient because it's stringy data in an SQL database, but I look forward to support tickets because i don't have to "debug". I just read the events, and have the evidence to back up that the customer is wrong and the system is right.

Re: Event Sourcing in Go: From Zero to Production

#47

Post by mrkeen »

zknill wrote: post #18
azkalam wrote: post #16

Earlier quoted context omitted.

- Write side is a Postgres INSERT - Read side is a SELECT on a Postgres view

I think you might struggle to "scale the read and write sides independently". It's a real stretch to be describing a postgres view as CQRS

I've heard worse, e.g.

You use POST for your Cs and GET for your Qs. Tada!

Re: Event Sourcing in Go: From Zero to Production

#48

Post by mrkeen »

mrsmrtss wrote: post #13
anthonylevine wrote: post #10

Earlier quoted context omitted.

> CQRS sounds good, until you just want to read a value that you know has been written. This is for you and the author apparently: Prating CQRS does not mean you're splitting up databases. CQRS is simply using different models for reading and writing. That's it. Nothing about different databases or projections or event sourcing. This quote from the article is just flat out false: > CQRS introduces eventual consistenc…

Yes, I don't know where the misconception that CQRS or Event Sourcing automatically means eventual consistency comes from. We have built, run, evolved, and operated quite a few reasonably sized event sourced systems successfully, and these systems are running to this day without any major incidents. We added eventually consistent projections where performance justified it, fully aware of the implications, but kept mo…

If you segregate your read and write, i.e put a (P)artition between them, and require writes to be (A)vailable when reads are down (and vice-versa), then you're either inconsistent, or eventually-consistent.

Re: Event Sourcing in Go: From Zero to Production

#49

Post by hu3 »

mrkeen wrote: post #45
simonw wrote: post #24

Earlier quoted context omitted.

This comment just made it finally click for me why event sourcing sounds so good on paper but rarely seems to work out for real-world projects: it expects a level of correct-design-up-front which isn't realistic for most teams.

> it expects a level of correct-design-up-front which isn't realistic for most teams. The opposite is true. A non-ES system is an ES system where you are so sure about being correct-up-front that you perform your reduce/fold step when any new input arrives, and throw away the input . It's like not keeping your receipts around for tax time (because they might get crinkled or hard to read, or someone might want to chan…

Most systems just need to be correct-enough-up-front though.

Re: Event Sourcing in Go: From Zero to Production

#50

Post by mrkeen »

liampulles wrote: post #28

If you are considering event sourcing, run an event/audit log for a while and see if that does not get you most of the way there. You get similar levels of historical insight, with the disadvantage that to replay things you might need to put a little CLI or script together to infer commands out of the audit log (which if you do a lot, you can make a little library to make building those one off tools quite simple - I…

I've done this.

We have a customer whom we bill for feature X.

Does he actually have feature X or are we billing him for nothing?

With ES: We see his Subscriptions and Cancellations and know if he has feature X.

Without ES: We don't know if he subscribed or cancelled.

With audit log: We almost know whether he subscribed or cancelled.

Post reply on HN