Live data from Hacker News

Engineering principles for building financial systems

substack.wasteman.codes

21–30 of 112 posts

Re: Engineering principles for building financial systems

#21

> Use integer representations of time This sounds like a type system issue (and the linked Etsy post is also a type system issue), or more precisely languages / systems with a loose type system where you can just pretend a time is a number without having to explicitly say what that number means. Conventions are good, but (real) type systems are better because they check everything every time.

Yes. And in this context, it doesn't even matter (too much) whether your type system is statically checked or done at runtime.

The main difference is whether the compiler will yell at you, or whether it'll be a runtime error. But the important point is that your mixing up won't go through and silently do the wrong thing.

Re: Engineering principles for building financial systems

#22

On the best practices, as noted by others, there are probably classes in your standard lib/ commons lib that cover this stuff better than storing in integers (e.g. BigDecimal in Java and Python Decimal have precision and rounding mode concepts built in). Something I've found valuable is on is managing euler units/ratios (e.g. proportions 10^1, percentages 10^-2, basis points 10^-4) . Enforcing a standard that all int…

You standard library might even already have libraries for eg storing currencies or time, too.

Re: Engineering principles for building financial systems

#23

Representing prices is difficult because a lot of financial instruments have widely different values and price increments. Decimal floating-point gives you the range to cover everything but is generally very inefficient to process. What you usually want is fixed-point decimal, with the caveat that the scaling factor will be different per asset.

Depending on how many numbers you process, inefficiency might not matter.

You could probably use arbitrary precision rational numbers (eg as implemented in Python https://docs.python.org/3/library/fractions.html) for all your accounting needs, and it would still be fast enough.

Re: Engineering principles for building financial systems

#25
post #3

> Use consistent rounding methodologies This may also mean promoting them to a named part of the business-domain in code, with their own functions, unit-tests, stuff like "fetch Rounding Strategy Suite by country code", etc. > Use integer representations of time. This one is a little controversial but I stand by it. There are so many libraries in different technologies that parse timestamps into objects, and they all…

> Unix timestamp, or even integer based UTC datetimes work perfectly fine.

For serious work, it’s worth noting that leap seconds are not representable in this format. Many financial applications can get away without representing leap seconds, but this is fundamentally a kludge. If you actually need to represent any point in time (according to UTC or any particular timezone), use a representation that actually represents what you want.

This is especially true for future times. If you mean “10 AM, August 1, 2025, New York time”, then represent that! Do not kludge it as an integer, do not use a different time zone as a proxy, do not use UTC, and don’t use a representation that contains the terms “standard” or “daylight”. If you do this wrong, you will eventually regret it. Bonus points for also being able to represent future leap seconds, I suppose, but I’ve never encountered any need to do that.

Keep in mind that those properly represented future times are not guaranteed to ever happen. Time zone definitions can and do change.

(And, since this is finance, you also get to deal with bank holidays, past, present, and future. Your legal department may have opinions about what should happen when a future date, that you thought wouldn’t be a bank holiday, turns out to be a bank holiday after all.)

(Also, since this is finance, sometimes people will want to translate a time into a date. This can be a nontrivial operation. The date associated with a timestamp may or may not matter for actual business purposes, and various parties may or may not agree about what date a certain time belongs to and may or may not implement their opinion correctly.)

Re: Engineering principles for building financial systems

#26
> Use integers to represent financial amounts

I followed this mantra when building a trading system. This was the worst engineering decision I ever made - it added complexity and runtime overhead, plus readability was reduced when debugging.

If your language has a proper decimal type like BigDecimal in Java or Decimal in Swift, I'd suggest using it. They offer perfect precision for financial applications, are battle tested and are a natural way to represent financial amounts. Yes, Java BigDecimal is unweildy, but it works.

If you're using JavaScript however, definitely use integers to represent financial amounts.

Re: Engineering principles for building financial systems

#27
post #4

I wouldn't endorse many of the things stated there. A lot of the points are inaccurate or straight up misleading. > The three main goals of your accounting system are to be (1) Accurate, (2) Auditable and (3) Timely. You forgot "consistent", which is very different from "accurate". Most financial transactions have multiple time dimensions: a trade is requested, booked, filled, matched, reconciliated, paid, settled at…

> Most financial transactions have multiple time dimensions: a trade is requested, booked, filled, matched, reconciliated, paid, settled at different times. These times often don't even follow a perfectly logical order (you can request and fill a trade and book it an hour later).

Similar issues exist with transaction identifiers. Every party involved with any aspect of a transaction may assign that aspect of the transaction zero or more identifiers, those identifiers may or may not be unique over any particular set of transactions and/or events related to transactions, and the sources of those identifiers may or may not even correctly document what, exactly, those identifiers identify. (And their representatives may think they know, and they may be wrong.) Your own code may need to generate identifiers for events associated with transactions, and you may or may not want to leak information in your choice of identifiers.

For new designs, my suggestion would be to treat these as one-to-many relations. Have a table along the lines of (transaction id, event, identifier issuer, identifier name, identifier). You might also need a timestamp in there, and you may need to throw in a row id of some sort as well. You might want to make an allowance for identifiers having variable type — are they bytes? Unicode? something else? You could plausibly get away with making these be JSON keys in an event object instead, but this gets awkward when someone tells you an identifier after the fact.

Re: Engineering principles for building financial systems

#28
post #24

A great companion read is Martin Fowler’s “Accounting Patterns”[1]. Having built and maintained systems that manage financial events for over a decade, I wish I had read these patterns earlier. [1] https://martinfowler.com/apsupp/accounting.pdf

I think Fowler's work is an underrated must-read for anyone who works in domains related to moving money. Makes any kind of engineering practices and architecture principles logical and make sense.

Re: Engineering principles for building financial systems

#29
We're basically talking about bookkeeping systems here, rather than high finance. Use a good relational database.

1. ACID, so you don't have to invent it.

2. Arbitrary precision numeric data types with vetted operations and rounding modes.

3. It does time as well as anyone can

4. Your computations and reporting can be done entirely in SQL.

5. When you get good at SQL or hire someone who is, the reporting is elegant.

6. It's as fast as it needs to be, and a good database architect can make it extremely fast.

7. It has all of the tooling to do disaster prevention and recovery.

I've built so many financial systems for the biggest multinationals in the world, and I have never regretted doing the bulk of the work in the database. Coca-Cola, GE, UPS, AIG, every iteration of the Tycos, basically every pharma with their crazy transfer pricing schemes... Whenever I experienced a performance problem, there have been way to address it that were far easier than what would have been to reinvent all of the hard tech that underlies consistent, accurate financial systems.

Re: Engineering principles for building financial systems

#30
Regarding the types, I think it's easy to fall into the trap of thinking you have to try and force the domain concept into one of the commonly available data structures at the point of creation, i.e. the price "£3.99" has to become (int) 399 or (float) 3.99000000001.

Actually, as another commentor mentioned, you can and should just choose something that represents what you want. In many cases it will be an object with various domain-specific fields. If you need to be sure that "3.99" always comes back as "3.99", you can use a string, which doesn't have any issues with precision in the storage format, and will usually map exactly to how humans deal with the concept. (Not necessarily recommending this, but it's worth considering.)

Similarly with JavaScript precision issues, to take another example that's been mentioned here - if you need numbers to be precise and the performance cost is acceptable, JavaScript can compute to arbitrary precision just as well as any other language. Just don't assume you're forced to convert your concept of a number to use the built-in numeric types and operations.

Post reply on HN