Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fintech Engineering Handbook

w.pitula.me

Post reply on HN
Print view View original

51–60 of 234 posts

Re: Fintech Engineering Handbook

#51

Post by microgpt »

noitpmeder wrote: post #40
lxgr wrote: post #8

Word of advice to anyone considering the "minor-units precision" strategy for representing monetary amounts: Don't (or at least, don't use it as an interchange/API data format). It seems like a clever idea (fast integer math, no rounding problems for addition and subtraction), but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of…

The only real correct solution here is to send mantissa and exponent as two separate integers. It's trivial to convert between exponents for whatever math you want, it can be as correct as you want, and is unambiguous. In the HFT space you save some wire space if you can commit to a consistent exponent for some {slice} up front (think instrument/tick-size/asset-class/exchange/feed/server/whatever/...) such that you o…

If you do that though aren't you just reinventing floating-point?

Re: Fintech Engineering Handbook

#52

Post by microgpt »

denismenace wrote: post #10
lxgr wrote: post #8

Word of advice to anyone considering the "minor-units precision" strategy for representing monetary amounts: Don't (or at least, don't use it as an interchange/API data format). It seems like a clever idea (fast integer math, no rounding problems for addition and subtraction), but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of…

> but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of digits for a given currency Why would that be a problem? You just transform the values when interacting with their API.

Customer was charged $0.995 after fees, how to represent in your data model with integer cents?

Re: Fintech Engineering Handbook

#53

Post by jdw64 »

As a programmer, what I feel when I see fintech programmers each speaking from their own different experiences and perspectives is that it makes me wonder what it really means to be good at programming.

What user xlii said about not storing monetary amounts as floats is a common IEEE 754 issue. And while it's true that financial tracking should be done through immutable logs or event-based records, I don't think every surrounding service needs to be built with event sourcing. I think it's enough to apply it only to core logic like ledgers, settlements, orders, and executions. Looking at xlii's comment, it seems like a technique that only becomes viable when the modeling is successful.

User lxgr's comment points out that it's a minor-unit issue. If JSON numbers are parsed as floats by the language or parser, precision can be lost. Usually people send values with a separate decimal places field. However, I've heard that in HFT, they don't do that because the overhead itself is too costly.

And antonymoose's comment aligns with what many books say. That's why designs like this are common in FX or API contexts. It feels like protocol design, doesn't it?

Putting it all together, everyone's right within their own domain. While I think it'd be great to have someone like xlii as my senior programmer, I also feel like I wouldn't be able to design such a complex system myself. In that sense, everyone's statements are valid, and it's interesting to see how opinions diverge depending on the domain. Is this what expertise looks like

Looking at all this, it seems like you can roughly infer where a programmer is coming from based on their experience. Sometimes programming doesn't feel like finding the right answer, but more like choosing a worldview

Watching how programmers model their domains on HN is always fascinating. Sometimes I click on their profiles and add their domain knowledge to my own personal wiki, thinking I might use it someday

Re: Fintech Engineering Handbook

#54

Post by lxgr »

noitpmeder wrote: post #40
lxgr wrote: post #8

Word of advice to anyone considering the "minor-units precision" strategy for representing monetary amounts: Don't (or at least, don't use it as an interchange/API data format). It seems like a clever idea (fast integer math, no rounding problems for addition and subtraction), but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of…

The only real correct solution here is to send mantissa and exponent as two separate integers. It's trivial to convert between exponents for whatever math you want, it can be as correct as you want, and is unambiguous. In the HFT space you save some wire space if you can commit to a consistent exponent for some {slice} up front (think instrument/tick-size/asset-class/exchange/feed/server/whatever/...) such that you o…

> The only real correct solution here is to send mantissa and exponent as two separate integers.

That’s essentially the same thing as a String-serialized big decimal, just less readable, no?

Re: Fintech Engineering Handbook

#55

Post by lxgr »

portly wrote: post #32
ivanmontillam wrote: post #16

Earlier quoted context omitted.

A string type. As parent says: it completely bypasses the problem. Save the numbers between double quotes and be done with it.

Storing numbers as arrays of u8? That doesn't make sense

It makes a lot of sense if you value correctness over performance.

Re: Fintech Engineering Handbook

#56

Post by lxgr »

notpushkin wrote: post #47
ivanmontillam wrote: post #41

Earlier quoted context omitted.

For JSON serialization, which doesn't support fixed-point precision it does. Floating-point precision has too many gotchas for being suitable to store Decimal types, especially for the Currency use case.

Surely it does: { "price": { "amount": 1000, "decimal_places": 2, "currency": "USD" } }

How is that better than {“amount”: “10.00”} (which also bypasses all potential floating point parsing issues that your or your counterparty’s JSON library might have)?

Re: Fintech Engineering Handbook

#57

Post by lxgr »

krever wrote: post #43
lxgr wrote: post #23

Earlier quoted context omitted.

You'll definitely have to throw it away at some point. The art is in making those points well-defined and rare enough to not cause large discrepancies, but frequent enough to avoid ballooning arbitrary-precision numbers across databases and services that might not be able to handle them.

I really like that phrasing! Would you mind if I steal in some form if I decide to review this part of the book?

Not at all, and thanks for writing all of this up!

Re: Fintech Engineering Handbook

#58

Post by belmarca »

Nice. The book contains a bunch of good information that could already be found elsewhere but collecting it is quite practical. I highly suggest to read Kleppmann's Designing Data-Intensive Applications. The first edition was very good, a second one came out recently.

I was CTO of a FinTech where I built the whole software stack from scratch: the lessons in the book are mostly correct. I say mostly, because as always, there is a lot of "it depends" to take into consideration for your particular project. For example, I chose to not use event-sourcing to avoid the whole state computation issue. A standard append-only audit trail can do the job.

You can't guarantee exactly-once delivery but you can construct effectively-once processing, and that is what you really want.

Store every request and response : absolutely, and not only when consuming APIs, but when collecting any information from the outside world (and, if you can, also log every intermediate transformation step within your perimeter). Content-adressed buckets + a relational table are great for this.

The text also does not mention anything about data lineage. What happens if a vendor updates some data mid-day that you absolutely need to be aware of? You need to be able to account for that, while also re-playing computations that used the old values and get the same result. It's not a particularly hard problem to solve, but it takes some thought.

Re: Fintech Engineering Handbook

#60

Post by jappgar »

benashford wrote: post #22

I think most of this applies to software engineering generally, not just fintech. For example the parts talking of retries, idempotency, event ordering, etc. This applies to all systems that require any degree of accuracy, even if no money is directly involved. I've seen so many systems built on the assumption that "we can always retry", but you can only retry if you fail cleanly in the first place, and if the downst…

I agree. Very little in here specifically applies to fintech except the ledgering and rounding parts, which are pretty light.

I would prefer to read a defense of something more radical like "database per account." Something that has unique tradeoffs within fintech.

Also, the main advice I would give to fintech engineers/founders is to take risk and compliance seriously from day one.

Financial systems are based around trust. If you don't provably mitigate risks you will lose trust and, eventually, your entire business.

Post reply on HN