Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fintech Engineering Handbook

w.pitula.me

Post reply on HN
Print view View original

81–90 of 234 posts

Re: Fintech Engineering Handbook

#81

Post by lxgr »

lawlorino wrote: post #72
lxgr wrote: post #69

Earlier quoted context omitted.

> thank you Rust decimals represented as JSON floats What do you mean? JSON doesn’t have floats, it has numbers, and how they’re used after being parsed is not part of the spec. > If I ever see a monetary value stored in something else than integers I'm going to run away screaming That’s good, then we’ll likely not be working on the same system :) I consider running from “amounts as integer” systems these days (but u…

> I consider running from “amounts as integer” systems these days (but usually unfortunately can’t). In the context of Fintech, how do you otherwise resolve floating point rounding issues if not representing amounts with integers?

Native decimal types, if your system has them. Many languages and databases used in financial contexts do.

Re: Fintech Engineering Handbook

#82

Post by gib444 »

lxgr wrote: post #8

Word of advice to anyone considering the "minor-units precision" strategy for representing monetary amounts: Don't (or at least, don't use it as an interchange/API data format). It seems like a clever idea (fast integer math, no rounding problems for addition and subtraction), but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of…

What is with this Twitter esque style of discussion? Post some vague comment with no real stake in the ground, but just reply to follow ups asking for clarifications about the right way. It's exhausting. Why not put all that effort into the initial comment?

Vague-posting seems to becoming more popular

Re: Fintech Engineering Handbook

#83

Post by lxgr »

traceroute66 wrote: post #80
lxgr wrote: post #78

Earlier quoted context omitted.

> Keeping the financial data whilst trashing the customer names, addresses etc. instantly on-demand before the expiry of the relevant time periods [...] Where does TFA recommend that? As I see it, it recommends separating PII data you'll eventually have to delete from that you'd probably want to keep forever (including data factoring into your accounting equations/invariants), so that you can delete the former after…

> any ideas and practices presented Unless its your job to architect stuff, in a financial firm you don't go looking around for ideas and practices. You comply with your employer's practices end of story. If you like looking up ideas and other people's practices then a heavily regulated environment is probably not the place for you. > how does one get there without having a conversation "having a conversation" about…

"Not thinking, just complying" isn't the panacea for good outcomes you make it out to be. You definitely want to limit the amount of excitement, but I've seen many issues caused by legacy formats and practices as well.

> You comply with your employer's practices end of story.

What if you're the employer ("first engineer" etc.), and there are no practices yet? Fintech almost by definition sometimes includes doing things from scratch because some existing solution or incumbent organization isn't working that well anymore.

> Unless its your job to architect stuff

Which seems to be the target audience/scenario for TFA.

Re: Fintech Engineering Handbook

#84

Post by lxgr »

jameshart wrote: post #62
lxgr wrote: post #56

Earlier quoted context omitted.

How is that better than {“amount”: “10.00”} (which also bypasses all potential floating point parsing issues that your or your counterparty’s JSON library might have)?

It is explicit about the fact that that number of decimal places is part of the data. The semantics for your string “10.00” are complex - is it considered equal to “10”? To “10.000”? To “10.001”? A user interacting with an API that uses such a string might make all sorts of assumptions about what it supports. A user interacting with an API that has an explicit decimal places concept is being told ‘decimals matter! Th…

> The semantics for your string “10.00” are complex - is it considered equal to “10”?

Yes, but "10 USD" would be a non-canonical representation and you probably serialized incorrectly.

> To “10.000”?

Yes, but same caveat as above applies.

> To “10.001”?

Obviously not, and any system you'd ever want to use in a financial context will tell you so.

Re: Fintech Engineering Handbook

#85

Post by lxgr »

microgpt wrote: post #52
denismenace wrote: post #10

Earlier quoted context omitted.

> but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of digits for a given currency Why would that be a problem? You just transform the values when interacting with their API.

Customer was charged $0.995 after fees, how to represent in your data model with integer cents?

You'll have to decide when and how to round. Keeping individual billing items at high precision and rounding after summing them up can work; defining and documenting a rounding policy (or complying with whatever's legally required in your jurisdiction/domain) and rounding each individual billed item can as well.

Re: Fintech Engineering Handbook

#86

Post by gmm1990 »

lxgr wrote: post #54
noitpmeder wrote: post #40

Earlier quoted context omitted.

The only real correct solution here is to send mantissa and exponent as two separate integers. It's trivial to convert between exponents for whatever math you want, it can be as correct as you want, and is unambiguous. In the HFT space you save some wire space if you can commit to a consistent exponent for some {slice} up front (think instrument/tick-size/asset-class/exchange/feed/server/whatever/...) such that you o…

> The only real correct solution here is to send mantissa and exponent as two separate integers. That’s essentially the same thing as a String-serialized big decimal, just less readable, no?

That’s quite a bit slower to process. At least if you’re converting to integers to do the calculations and the calculations would be quite a bit slower if you kept the big decimal type

Re: Fintech Engineering Handbook

#87

Post by andylynch »

afavour wrote: post #65
denismenace wrote: post #10

Earlier quoted context omitted.

> but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of digits for a given currency Why would that be a problem? You just transform the values when interacting with their API.

Because a lot of the time there won’t be any error when you’re wrong, just silent data loss.

I’ve seen bugs like this in prod systems. The notional value of the error tends to make the people concerned anything but silent.

Re: Fintech Engineering Handbook

#88

Post by lxgr »

gmm1990 wrote: post #86
lxgr wrote: post #54

Earlier quoted context omitted.

> The only real correct solution here is to send mantissa and exponent as two separate integers. That’s essentially the same thing as a String-serialized big decimal, just less readable, no?

That’s quite a bit slower to process. At least if you’re converting to integers to do the calculations and the calculations would be quite a bit slower if you kept the big decimal type

True, but this is usually your least concern when you're dealing with monetary amounts/math.

Re: Fintech Engineering Handbook

#89

Post by foresterre »

lxgr wrote: post #79
morpheuskafka wrote: post #76

Earlier quoted context omitted.

I'm sure there is something I don't know here, but how is working with integers "brittle"? The only issue I see is rounding down by default, not sure if that is even an issue or not. At any rate, it seems a lot less brittle than floats or bigdecimal style number classes.

The brittleness comes from the fact that the number of implied decimal digits per currency isn't always well-defined across all stakeholders and systems. If you're only working in a single currency, there's usually no issue.

As a general rule, you always include the currency code (EUR, SEK, USD etc.) and if possible also the amount of decimals, when using minor units.

Currency codes can be found in ISO 4217.

Re: Fintech Engineering Handbook

#90

Post by lxgr »

gib444 wrote: post #82
lxgr wrote: post #8

Word of advice to anyone considering the "minor-units precision" strategy for representing monetary amounts: Don't (or at least, don't use it as an interchange/API data format). It seems like a clever idea (fast integer math, no rounding problems for addition and subtraction), but it'll bite you incredibly hard if you ever stumble upon an edge case such as working with a partner that has a different implied number of…

What is with this Twitter esque style of discussion? Post some vague comment with no real stake in the ground, but just reply to follow ups asking for clarifications about the right way. It's exhausting. Why not put all that effort into the initial comment? Vague-posting seems to becoming more popular

If there were a simple one-size-fits-all solution to these problems, there wouldn't be a need for a handbook, nor for a discussion, would there?

I can't design everybody's systems here, but I was hoping that sharing some war stories that have cost me days or weeks of work might sensitize somebody to a few non-obvious footguns.

Post reply on HN