Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing (2017)

arkwright.github.io

Post reply on HN
Print view View original

11–20 of 80 posts

Re: Event Sourcing (2017)

#11

Post by imglorp »

guywhocodes wrote: post #9
tunesmith wrote: post #3

Earlier quoted context omitted.

I saw the Akka people talking about this on twitter once, I think they were theorizing that encrypting the data in the log would be sufficient, because then "deleting the key" could be interpreted as the deletion of the record (even though the useless data still exists in the log). But I'm not sure this was ever legally validated?

That is a very interesting approach, however I can imagine that it can be quite a large database of encryption keys. I'll have to build a small system for this and try it.

Oh, why? Holding one key for every human on earth would fit in 8 TB plus read-mostly replicas. You'd delete their key if they made GDPR removal request and the keyless, encrypted, immutable entries would be entombed in place.

Re: Event Sourcing (2017)

#12

Post by xtagon »

tunesmith wrote: post #3
xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

I saw the Akka people talking about this on twitter once, I think they were theorizing that encrypting the data in the log would be sufficient, because then "deleting the key" could be interpreted as the deletion of the record (even though the useless data still exists in the log). But I'm not sure this was ever legally validated?

That is what I was referring to as "crypto-erasure" (also known as "crypto-shredding"). I'm not sure what counts as legally validated, but some have shared concerns that the encryption you use to do this would need to be future-proof against, say, advancements in quantum computing cracking the encryption down the road even after the key has been thrown out.

Re: Event Sourcing (2017)

#13

Post by rodocite »

guywhocodes wrote: post #8
ec109685 wrote: post #5

This article was good. It didn’t get into read after write consistency well enough in my opinion, which breaks the event sourcing pattern for many use cases. E.g. in the create user example, there are states in the system where a user could create an account and then reload their page and have the account not be there if the write hasn’t propagated to the database used to satisfy reads.

While I'm no expert in the subject. Shouldn't read after write in event sourcing be nonsensical in terms of correctness? Instead you have to convert that problem into asynchronously waiting for a ACK message that your message had its intended effect. And only then would you ask to read the state? EDIT: This of course does not cover the optimistic concurrency models in say PostgreSQL where you can effectively begin-wr…

I don't quite understand this comment. Are you looking for a confirmation? Is it good enough to just have the most updated data for the state you want to track? I'm curious about what the specific use case was.

if you are waiting for ACK, you could alternatively create another stream that informs you of whatever write you were waiting for and therefore pushes you the most updated value or triggers the read. you can use the correlationID to make sure it's the set of changes that are bound to the original event you're tracking.

the "correctness" issue will crop up in whatever model you use since concurrency is the primary means for scaling a system. wrong order writes will have "correctness" problems with any db.

if you need strict ordering, the solution in ES will probably be the same as in pg-- separate the streams that need to be ordered and optimize them up front as much as you can. then post process when you no longer can't.

if data corruption is the issue, a changeset validation prior to the write probably works better than a rollback. if the validation is bad, catch it and send it to an exception stream you can track. event sourcing allows you to track by event/causation/correlation ids so you'll probably have an easier time debugging that.

Re: Event Sourcing (2017)

#14

Post by nchase »

rodocite wrote: post #4
xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

I'm pretty sure you can "soft-delete" for GDPR compliance. So this concern is sort of a non-issue. Besides that: 1. If you're using an immutable structure, as long as you use references, you can obfuscate data. Blockchains ran into this problem before GDPR requirements and that's essentially all they do. 2. ^ "Update" strategy for event sourcing is the same as above. Essentially a copy of the log or the log slice the…

Can you describe more precisely how you're defining "soft-delete" here?

Re: Event Sourcing (2017)

#15

Post by rodocite »

xtagon wrote: post #12
tunesmith wrote: post #3

Earlier quoted context omitted.

I saw the Akka people talking about this on twitter once, I think they were theorizing that encrypting the data in the log would be sufficient, because then "deleting the key" could be interpreted as the deletion of the record (even though the useless data still exists in the log). But I'm not sure this was ever legally validated?

That is what I was referring to as "crypto-erasure" (also known as "crypto-shredding"). I'm not sure what counts as legally validated, but some have shared concerns that the encryption you use to do this would need to be future-proof against, say, advancements in quantum computing cracking the encryption down the road even after the key has been thrown out.

if the references don't point to actual data, you don't need this.

- minimum 2 parts. a relay (reference hash) and the cold/true storage portion. you can break the reference hash up and reassemble only for secret holders. which brings us to:

- content-based routing

- there are also deterministic vaults for rolling keys

i'm actually not sure in what high-level situation "crypto-erasure" would work in because being able to re-key a reference means you have complete control. so why would you need to erase the "bad" key when you can just switch the reference?

Eth draft for enabling cold storage relay https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1077.m...

Decentralized ID https://www.w3.org/TR/did-core/

^ both generally use the same concept i described above and solve GDPR "delete" issue. actually, it solves GDPR completely if you can just rely on the DID. "hard delete" is a separate issue, though-- no other way to get around that but to fork/version + replay your store and re-reference anyone who wants to hard delete if you didn't use reference hashes.

Re: Event Sourcing (2017)

#16

Post by rodocite »

nchase wrote: post #14
rodocite wrote: post #4

Earlier quoted context omitted.

I'm pretty sure you can "soft-delete" for GDPR compliance. So this concern is sort of a non-issue. Besides that: 1. If you're using an immutable structure, as long as you use references, you can obfuscate data. Blockchains ran into this problem before GDPR requirements and that's essentially all they do. 2. ^ "Update" strategy for event sourcing is the same as above. Essentially a copy of the log or the log slice the…

Can you describe more precisely how you're defining "soft-delete" here?

data is still stored somewhere but any routing to the data is disabled and the entity is disassociated.

Re: Event Sourcing (2017)

#17

Post by pookeh »

imglorp wrote: post #11
guywhocodes wrote: post #9

Earlier quoted context omitted.

That is a very interesting approach, however I can imagine that it can be quite a large database of encryption keys. I'll have to build a small system for this and try it.

Oh, why? Holding one key for every human on earth would fit in 8 TB plus read-mostly replicas. You'd delete their key if they made GDPR removal request and the keyless, encrypted, immutable entries would be entombed in place.

What do you do when the data involves multiple people?

Re: Event Sourcing (2017)

#18

Post by codebje »

ec109685 wrote: post #5

This article was good. It didn’t get into read after write consistency well enough in my opinion, which breaks the event sourcing pattern for many use cases. E.g. in the create user example, there are states in the system where a user could create an account and then reload their page and have the account not be there if the write hasn’t propagated to the database used to satisfy reads.

That's a distributed systems problem, rather than an event sourcing one. I'm sure we've all done something like comment on a site like HN and not seen our comment appear when we reload. The more distributed the system, the more likely it is we're hitting a stale cache somewhere.

Even the most absurdly reduced system running on a single machine, taking an HTTP request in and processing it fully to completion in all aspects before returning any response, is a distributed system - the browser is at the other end, running asynchronously. The user may tell the browser to reload before the single server has finished processing. When do both the user and the server agree that the account has been created?

To "fix" the problem with event sourcing, just don't add distributed components if you don't need them. Synchronise your "on event" action handlers with your event creation, and don't return success to the command until the handlers have completed.

You can even choose to wrap it all in a transaction so the event doesn't write unless the handlers all succeed, side-stepping the problem of desynchronised views due to handler bugs.

You still keep the (IMO) main benefit of event sourcing: you can define new views you didn't have to foresee and build them from the complete history of the system as if you'd known about them from the start.

Re: Event Sourcing (2017)

#19

Post by codebje »

xtagon wrote: post #2

Great article. One thing that wasn't pointed out that might be of interest to someone learning about Event Sourcing is that it introduces some challenges if you are to be compliant with GDPR and similar laws. For example, if your event log is immutable, and you use it as an audit log, then by nature you are not ever deleting data. There are solutions to this (for example, crypto-erasure), but it can be non-trivial to…

Your event log should IMO be nominally immutable rather than actually immutable.

You should feel free to take actions such as expunging private or sensitive data as appropriate. Keep the events, but rewrite them to contain only the desired data. Trivial to implement, and simple.

I'd only worry about stuff like crypto-erasure if you physically cannot alter the past, such as if you have a requirement for non-repudiation or some such. Doing it just for technical purity isn't worth the cost :-)

Re: Event Sourcing (2017)

#20

Post by lukev »

A word of caution for anyone considering an event-sourced architecture. I was on a large government project where the decision was made to use event sourcing, and it was disastrous. It ended up being a big contributor to several years of time and cost overruns.

The reason is that for event sourcing to work, you have to have a pretty good idea of your application's requirements up front. It's simply not conducive to agile development, compared with using a traditional DB. The requirements were constantly shifting, and we were constantly realizing that we had the wrong semantics or structure for various fields, or that assumptions we had made about the coupling of different types of data simply didn't hold. This led to a ton of rewriting and churn on the view code, and required constant decisions on how to handle existing data in the "old" format.

Some of the requirement churn even had ramifications for fundamental architectural characteristics such as support for atomic transactions, so there were several points at which we had to hack locking or other techniques to ensure consistency on top of the event sourcing approach. I do NOT recommend this, it turns everything into a huge mess.

The worst part is, ultimately, the data sizes ended up not being that big. We could have run the whole thing off of append-only tables in a single beefy Postgres instance.

Conclusion: if you are designing systems, you should definitely know what event sourcing is and the benefits it can provide. However, avoid it by default in favor of simpler more traditional models unless they are really infeasible for what you are trying to do. And then, lock down as many key requirements (at the very least, those around consistency and interop with other systems) as possible before charging ahead with implementation.

Post reply on HN