Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Event Sourcing, CQRS and Micro Services: Real FinTech Example

lukasniessen.medium.com

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#61

Post by mrkeen »

ff4 wrote: post #46

They already had PostgreSQL (with its strong ACID guarantees) in place, yet the design introduces eventual consistency via MongoDB for reads—without a compelling justification. A DBA could have optimized those PostgreSQL queries to single-digit milliseconds, avoiding the added sync overhead entirely. Instead, it feels like unnecessary complexity was layered onto a proven double-entry ledger approach.

* Most single-db deployments give up on ACID for performance reasons (see READ_COMMITTED)

* Even if you have ACID, it's not sufficient for distributed systems. Its guarantees will keep one node consistent with itself. No transactionality between the customers app and your db.

Maybe you're one bank with all the customers, but as soon as you want to talk to other banks, are you really going to share the one ACID instance? Who's the DBA?

Currently the state of fintech is 90% of devs being in denial that they're in a distributed system.

> proven double-entry ledger approach.

Yes. In that language, the ledger is the list of events. If today's devs were around 300 years ago they'd be calling for 'balances' instead of 'ledgers' because they're simpler.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#62

Post by torben-friis »

saxenaabhi wrote: post #59
torben-friis wrote: post #56

Semi related question: does anyone have experience introducing proper financial data handling (ledgers or other alternatives) in a fintech _after the fact_ ? As in, fixing things during a scaleup phase when business has been working for a while and the original improvised systems are breaking, but you can’t stop business to repair. Currently undergoing a similar project and would really appreciate any resource thrown…

Isn't it simple? Create events from past relational data and add them to the log?

Well, yes and no. We’ve got engineers on one side who store mutable relational data (orders, purchases, subscriptions, what have you) and on the other side accountants thinking in terms of accounts (we have such and such in account 705xx).

Mapping the two domains is the main issue, and how much the new system should reflect accounting movement of money vs the current engineering model or a completely different in between

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#63

Post by antonvs »

ff4 wrote: post #46

They already had PostgreSQL (with its strong ACID guarantees) in place, yet the design introduces eventual consistency via MongoDB for reads—without a compelling justification. A DBA could have optimized those PostgreSQL queries to single-digit milliseconds, avoiding the added sync overhead entirely. Instead, it feels like unnecessary complexity was layered onto a proven double-entry ledger approach.

> proven double-entry ledger approach.

A double-entry ledger is a combination of a process and a view that was mistaken for a data model centuries ago, and that mistake became embedded.

Fundamentally, you’re dealing with a sequence of events. The double-entry ledger is a particular result of processing those events - a view. There are many other useful views.

This is well understood in academic accounting. See e.g. https://en.wikipedia.org/wiki/Resources,_Events,_Agents for an alternative system that doesn’t make the same mistake.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#64

Post by mrkeen »

torben-friis wrote: post #62
saxenaabhi wrote: post #59

Earlier quoted context omitted.

Isn't it simple? Create events from past relational data and add them to the log?

Well, yes and no. We’ve got engineers on one side who store mutable relational data (orders, purchases, subscriptions, what have you) and on the other side accountants thinking in terms of accounts (we have such and such in account 705xx). Mapping the two domains is the main issue, and how much the new system should reflect accounting movement of money vs the current engineering model or a completely different in bet…

Your engineers are wrong. You're not going to be able to bridge that gap.

Here's a scenario: you've partnered with a credit card provider. They charge some money each month per card, and you pass that onto your customers who use the cards.

One day the partner sends you a 'card-cancelled' message. Have you built your system to accept that message unconditionally? Or did your engineers put in defensive code ("fail fast", assertions, status checks, db constraints) so that your system can reject that message?

Because that's how we've built our system at work. Our engineers are proud of something called "data integrity" that our almost-ACID (READ_COMMITTED) DB supposedly has. We won't move to events (listening to what actually happened) because we'd be giving up on pretending that our DB guarantees somehow correspond to what's going on in the real world.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#65

Post by mrkeen »

saxenaabhi wrote: post #6

> INSERT INTO events (account_id, type, amount, timestamp) VALUES (123, 'deposit', 100, NOW()) How would it work if they had to support intra system transfers? So one user balance should be withdrawn and another should get a deposit? That's not possible to do atomically with event sourcing right?

You're getting hung up on the "double" entries. That's mostly for reducing accidental and deliberate (fraud) calculation errors if you have humans crunching the numbers. The two entries can be in the same row if you like.

As for the inter-system transfers, It's not possible in the general case. You might not have a network cable between the two systems. And if you do, you run into CAP, two-generals, etc.

ACID is out of the question because the two parties don't share a DB. And if they did, some tech lead would turn down the acidity level for performance reasons.

The best you can do is intelligently interpret the information that has been given to you. That's all ledgers are: a list of things that some system knows. 'UPDATE CustomerBalance...' (CRUD) is not a fact, but 'Customer paid...' is a fact, and that's all event-sourcing is.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#66

Post by mrkeen »

coryvirok wrote: post #50

Not sure why there is so much hate on this thread. I found the post well written, insightful, and pragmatic. Having built systems that process billions of events and displayed results, triggered notifications, etc in real time (not RTOS level, I'm talking 1 or 2 seconds of latency) you absolutely need to separate reads and writes. And if you can trust db replication to be fast and reliable, you can indeed skip distri…

> Not sure why there is so much hate on this thread.

1) "Kafka is resume-driven-development" is a meme.

2) Devs are in denial about being in a distributed system, and think that single-threaded thinking (in proximity to a DB that calls itself ACID) leads to correct results in a distributed setting.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#67

Post by mrngm »

torben-friis wrote: post #62
saxenaabhi wrote: post #59

Earlier quoted context omitted.

Isn't it simple? Create events from past relational data and add them to the log?

Well, yes and no. We’ve got engineers on one side who store mutable relational data (orders, purchases, subscriptions, what have you) and on the other side accountants thinking in terms of accounts (we have such and such in account 705xx). Mapping the two domains is the main issue, and how much the new system should reflect accounting movement of money vs the current engineering model or a completely different in bet…

I'm not sure why orders/purchases/subscriptions would be seen as mutable. At some point in time, customer X decided to exchange money amount M for a subscription Y. This means the company is obliged to deliver subscription Y, because the customer engaged in a contract with the company, moved amount M to the company, and expecting to (regularly) receive Y. At renewal, the customer, again, needs to pay amount M in order to continue receiving Y.

The only mutable thing here would be the end date of said subscription, at which point the company no longer requires amount M from the customer, and the customer no longer receives Y.

Then on the accounting side, every time subscription Y renews, said customer in account 750xx needs to have its balance lowered by amount M, only to get increased again when they pay.

The only way to bridge this gap is to have the engineers know what accounting needs, and let them build the right infrastructure. In this [2018] [video] https://www.youtube.com/watch?v=KH0l8QqhzYk I recently watched, the speaker Rahul Pilani explains how Netflix organised their billing systems, and how all parts fit together. I'm not saying you should copy their infrastructure, but it doesn't hurt to look at a higher level how the business operates and what their accounting requirements are.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#68

Post by torben-friis »

mrngm wrote: post #67
torben-friis wrote: post #62

Earlier quoted context omitted.

Well, yes and no. We’ve got engineers on one side who store mutable relational data (orders, purchases, subscriptions, what have you) and on the other side accountants thinking in terms of accounts (we have such and such in account 705xx). Mapping the two domains is the main issue, and how much the new system should reflect accounting movement of money vs the current engineering model or a completely different in bet…

I'm not sure why orders/purchases/subscriptions would be seen as mutable. At some point in time, customer X decided to exchange money amount M for a subscription Y. This means the company is obliged to deliver subscription Y, because the customer engaged in a contract with the company, moved amount M to the company, and expecting to (regularly) receive Y. At renewal, the customer, again, needs to pay amount M in orde…

>I'm not sure why orders/purchases/subscriptions would be seen as mutable. At some point in time, customer X decided to exchange money amount M for a subscription Y.

Think for example about orders where a costumer bought three items and later cancelled one, the order value mutates as it is updated and at most we have a copy of the previous order state before price was updated (for some cases not even that).

If you think that’s not a good modeling for financial processes, well so do I, it’s the legacy we’re supposed to manage; moving out from that type of non ideal system to something more solid is what I’m researching.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#69

Post by serguzest »

torben-friis wrote: post #56

Semi related question: does anyone have experience introducing proper financial data handling (ledgers or other alternatives) in a fintech _after the fact_ ? As in, fixing things during a scaleup phase when business has been working for a while and the original improvised systems are breaking, but you can’t stop business to repair. Currently undergoing a similar project and would really appreciate any resource thrown…

We are in a similar situation, and here's what I did:

1) Learned the basic concepts of double-entry bookkeeping. 2) Told ChatGPT about my business domain and requested an example Chart of Accounts (CoA) tailored to it.

Feel free to reach out to me, I’d love to exchange ideas.

Re: Event Sourcing, CQRS and Micro Services: Real FinTech Example

#70

Post by mrngm »

torben-friis wrote: post #68
mrngm wrote: post #67

Earlier quoted context omitted.

I'm not sure why orders/purchases/subscriptions would be seen as mutable. At some point in time, customer X decided to exchange money amount M for a subscription Y. This means the company is obliged to deliver subscription Y, because the customer engaged in a contract with the company, moved amount M to the company, and expecting to (regularly) receive Y. At renewal, the customer, again, needs to pay amount M in orde…

>I'm not sure why orders/purchases/subscriptions would be seen as mutable. At some point in time, customer X decided to exchange money amount M for a subscription Y. Think for example about orders where a costumer bought three items and later cancelled one, the order value mutates as it is updated and at most we have a copy of the previous order state before price was updated (for some cases not even that). If you th…

I suppose that's a flexible way of looking at orders, but it sounds more like a shopping basket. What if the customer ordered three items, you started shipping those, then the customer cancels one item. That sounds like the wrong order (hah) of things.
Post reply on HN