Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing (2017)

arkwright.github.io

Post reply on HN
Print view View original

21–30 of 80 posts

Re: Event Sourcing (2017)

#21

Post by eweise »

xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

Don't stick PII in the events. Write it to a dedicated PII db and reference it in the event. Easy to get rid of the person's data. Just delete from the PII db.

Re: Event Sourcing (2017)

#22

Post by Yuioup »

lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

Thank you for this. I've been warning people off Event Sourcing for a while now. The architecture is the most convoluted, pretentious, redundant and downright soul-crushing. If you see Event Sourcing anywhere, run away. Run far, far away.

Re: Event Sourcing (2017)

#24

Post by lawik »

rodocite wrote: post #4
xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

I'm pretty sure you can "soft-delete" for GDPR compliance. So this concern is sort of a non-issue. Besides that: 1. If you're using an immutable structure, as long as you use references, you can obfuscate data. Blockchains ran into this problem before GDPR requirements and that's essentially all they do. 2. ^ "Update" strategy for event sourcing is the same as above. Essentially a copy of the log or the log slice the…

Soft delete in the sense of removing availability but keeping the data shouldn't be GDPR compliant from what I've seen. Not in regards to right to be forgotten and restrictions on keeping PII.

Re: Event Sourcing (2017)

#25

Post by andreareina »

lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

I'd love to read about the original assumptions -> original design -> revised assumptions -> design invalidations if you care (and are able) to write.

Re: Event Sourcing (2017)

#26

Post by andreareina »

codebje wrote: post #19
xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

Your event log should IMO be nominally immutable rather than actually immutable. You should feel free to take actions such as expunging private or sensitive data as appropriate. Keep the events, but rewrite them to contain only the desired data. Trivial to implement, and simple. I'd only worry about stuff like crypto-erasure if you physically cannot alter the past, such as if you have a requirement for non-repudiatio…

I think cases where you cannot alter the past (or can only do so with difficulty) are fairly common; backups will easily tend to fall in that category.

Re: Event Sourcing (2017)

#27

Post by AndrewKemendo »

Yuioup wrote: post #22
lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

Thank you for this. I've been warning people off Event Sourcing for a while now. The architecture is the most convoluted, pretentious, redundant and downright soul-crushing. If you see Event Sourcing anywhere, run away. Run far, far away.

I think you're doing it wrong. It shouldn't be any more complicated than writing events as a producer to a Kafka topic.

Gets more complicated when you add in distributed computing, redundancy etc... However those things are always complicated

Re: Event Sourcing (2017)

#28

Post by inopinatus »

Yuioup wrote: post #22
lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

Thank you for this. I've been warning people off Event Sourcing for a while now. The architecture is the most convoluted, pretentious, redundant and downright soul-crushing. If you see Event Sourcing anywhere, run away. Run far, far away.

Some of worlds most useful and powerful data structures are the projection of an event log. The tables of a RDBMS, the balanced writes of a SSD, and even the classic: double-entry book-keeping.

Even the data stream of a TCP connection is a projection of events, which is why (and how) we can reconstruct them by replaying captured segments.

So just because there are some lousy executions of a general architecture, doesn’t mean we should recoil from the basic idea.

My takeaway is that successful event-sourced structures are crafted for the domain they represent. I’ve developed a couple for my own work, for very specific aspects of an application, and they work well in context.

If your experience has been that a general-purpose ES framework leads to shitty, hard-to-maintain apps, I’d say that’s evidence for the corollary.

Re: Event Sourcing (2017)

#29

Post by AndrewKemendo »

lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

I mean that conclusion is true of basically every complex system. That's the whole point of hacking together an MVP first, to see what really matters.

However it's silly IMO to use that criteria as an example of why not to do event driven architecture. It's well understood in the eda community that you generally don't start with streaming systems unless you know you need it from the start.

Re: Event Sourcing (2017)

#30

Post by rolling_roland »

lukev wrote: post #20

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns. The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to a…

This, this and third time this. I have the exact same experience. You should know the business requirements very well before you choose event sourcing and hope they don't start changing drastically.
Post reply on HN