Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing, CQRS and Micro Services: Real FinTech Example

lukasniessen.medium.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#11

Post by saxenaabhi »

javcasas wrote: post #8
saxenaabhi wrote: post #6

> INSERT INTO events (account_id, type, amount, timestamp) VALUES (123, 'deposit', 100, NOW()) How would it work if they had to support intra system transfers? So one user balance should be withdrawn and another should get a deposit? That's not possible to do atomically with event sourcing right?

Create an event type for transfer with amount, source account and destination account.

Right, but then your downstream consumers have to be global and can't be partitioned based on account_id or some other partition key?

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#12

Post by javcasas »

nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

Yeah, that's a big NO. Events are immutable. If an event is wrong, you post an event with an amendment. Then yes, rebuild the app state.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#13

Post by [deleted] »

nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

[deleted]

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#14

Post by saxenaabhi »

javcasas wrote: post #12
nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

Yeah, that's a big NO. Events are immutable. If an event is wrong, you post an event with an amendment. Then yes, rebuild the app state.

Not speaking about their case, but I think some cases a "versioned mutable data store" with a event log that lists updates/inserts makes more sense than an "immutable event log" one like kafka.

Consider the update_order_item_quantity event in a classic event sourced systems. It's not possible to guarantee that two waiters dispatching two such events at same time when current quantity is 1 would not cause the quantity to become negative/invalid.

If the data store allowed for mutability and produced an event log it's easy:

Instead of dispatching the update_order_item_quantity you would update the order document specifying the current version. In the previous example second request would fail since it specified a stale version_id. And you can get the auditability benefits of classic event sourcing system as well because you have versions and an event log.

This kind of architecture is trivial to implement with CouchDB and easier to maintain than kafka. Pity it's impossible to find managed hosting for CouchDB outside of IBM.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#15

Post by zsoltkacsandi »

nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

Instead of modifying the original (and incorrect) event, you can add a manual correction event with the info of who did it and why, and replay the events. This is how we dealt with such corrections with event sourcing.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#16

Post by javcasas »

saxenaabhi wrote: post #11
javcasas wrote: post #8

Earlier quoted context omitted.

Create an event type for transfer with amount, source account and destination account.

Right, but then your downstream consumers have to be global and can't be partitioned based on account_id or some other partition key?

Consumers will have to read all events that concern them, yes. Maybe this type of event must end up in multiple partitions and will require some architectural shenanigans.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#17

Post by throwup238 »

javcasas wrote: post #12
nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

Yeah, that's a big NO. Events are immutable. If an event is wrong, you post an event with an amendment. Then yes, rebuild the app state.

The big caveat here is GDPR and other privacy laws. In some cases you need the ability to scrub the event store completely of PII for legal reasons, even if only to null the relevant fields in the events.

Without preemptive defensive coding in your aggregates (whatever you call them) this can quickly blow up in your face.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#18

Post by javcasas »

saxenaabhi wrote: post #14
javcasas wrote: post #12

Earlier quoted context omitted.

Yeah, that's a big NO. Events are immutable. If an event is wrong, you post an event with an amendment. Then yes, rebuild the app state.

Not speaking about their case, but I think some cases a "versioned mutable data store" with a event log that lists updates/inserts makes more sense than an "immutable event log" one like kafka. Consider the update_order_item_quantity event in a classic event sourced systems. It's not possible to guarantee that two waiters dispatching two such events at same time when current quantity is 1 would not cause the quantity…

Any modern DB with a WAL (write ahead log) is an immutable event system, where the events are the DB primitives (insert, update, delete...).

When you construct your own event system you are constructing a DB with your own primitives (deposit, withdraw, transfer, apply monthly interest...).

You have to figure out your transaction semantics. For example, how to reject invalid events.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#19

Post by javcasas »

throwup238 wrote: post #17
javcasas wrote: post #12

Earlier quoted context omitted.

Yeah, that's a big NO. Events are immutable. If an event is wrong, you post an event with an amendment. Then yes, rebuild the app state.

The big caveat here is GDPR and other privacy laws. In some cases you need the ability to scrub the event store completely of PII for legal reasons, even if only to null the relevant fields in the events. Without preemptive defensive coding in your aggregates (whatever you call them) this can quickly blow up in your face.

What I have read about it is: encrypt PII with a client-depending key, do not post the key to the event system. When an erasure request comes in, delete the corresponding key. Now the data cannot be decrypted anymore for that client.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#20

Post by speed_spread »

nateroling wrote: post #5

This made me do a double-take. Surely you would never do this, right? It seems to be directly counter to the idea of being able to audit changes: “Event replay: if we want to adjust a past event, for example because it was incorrect, we can just do that and rebuild the app state.”

It's poorly phrased but I'm not sure they meant "mutate the past". The keyword is "adjust" which could mean "append a correction".
Post reply on HN