Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of Fast-Math

simonbyrne.github.io

Post reply on HN
Print view View original

141–150 of 233 posts

Re: Beware of Fast-Math

#141

Post by Filligree »

scott_w wrote: post #96
simonw wrote: post #80

Earlier quoted context omitted.

I've been having an interesting challenge relating to this recently. I'm trying to calculate costs for LLM usage, but the amounts of money involved are so tiny . Gemini 1.5 Flash 8B is $0.0375 per million tokens! Should I be running my accounting system on units of 10 billionths of a dollar?

Fixed point Decimal is your friend here. I’m guessing you buy tokens in increments of 1,000,000 so it isn’t too much of an issue to account for. You can then normalise in your accounting so 1,000,000 is just “1 unit,” or you can just account in increments of 1,000,000 but that does start looking weird (but might be necessary!)

No, billing happens per-token. It’s entirely necessary to use billionths of a dollar here, if you don’t use floating point.

Re: Beware of Fast-Math

#142

Post by Izkata »

krapht wrote: post #127
nulld3v wrote: post #116

Earlier quoted context omitted.

I'm curious where you got this idea from because it is trivially disprovable by typing 0.1 or 0.01 into any python or JS REPL?

https://docs.python.org/3/tutorial/floatingpoint.html Stop at any finite number of bits, and you get an approximation. On most machines today, floats are approximated using a binary fraction with the numerator using the first 53 bits starting with the most significant bit and with the denominator as a power of two. In the case of 1/10, the binary fraction is 3602879701896397 / 2 * 55 which is close to but not exactly…

For anyone curious about testing it themselves and/or wanting to try other numbers:

  >>> from decimal import Decimal
  >>> Decimal(0.1)
  Decimal('0.1000000000000000055511151231257827021181583404541015625')

Re: Beware of Fast-Math

#143

Post by malfist »

gamescr wrote: post #137
usefulcat wrote: post #122

Earlier quoted context omitted.

You can certainly make trading systems that work using floating point, but there are just so many fewer edge cases to consider when using fixed point. With fixed point and at least 2 decimal places, 10.01 + 0.01 is always exactly equal to 10.02. But with FP you may end up with something like 10.0199999999, and then you have to be extra careful anywhere you convert that to a string that it doesn't get truncated to 10.…

I work on game engines and the problem with floats isn't on small values like 10.01 but on large ones like 400,010.01 that's when the precision wildly varies.

Not only that but the precision loss accumulates. Multiply too many numbers with small inaccuracies and you wind up with numbers with large inaccuracies

Re: Beware of Fast-Math

#145

Post by rendaw »

jcranmer wrote: post #132
T0Bi wrote: post #130

Earlier quoted context omitted.

> Using some kind of fixed point math would be entirely inappropriate for most HFT or scientific computing applications. May I ask why? (generally curious)

For starters, it's giving up a lot of performance, since fixed-point isn't accelerated by hardware like floating-point is.

Isn't fixed point just integer?

Re: Beware of Fast-Math

#146

Post by pgwhalen »

jcranmer wrote: post #129
razighter777 wrote: post #65

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

I've found fear of the use of floating-point in finance to be a good litmus test for how knowledgeable people are about floating-point. Because as far as I can tell, finance people almost exclusively uses (binary) floating-point [1], whereas a lot of floating-point FUD focuses on how disastrous it is for finance. And honestly, it's a bit baffling to me why so many people seem to think that floating-point is disastrou…

I agree overall but my take is that it shows more ignorance about the domain of finance (or a particular subdomain) than it does about floating-point ignorance.

It’s really more of a concern in accounting, when monetary amounts are concrete and represent real money movement between distinct parties. A ton of financial software systems (HFT, trading in general) deal with money in a more abstract way in most of their code, and the particular kinds of imprecision that FP introduces doesn’t result in bad business outcomes that outweigh its convenience and other benefits.

Re: Beware of Fast-Math

#147

Post by munch117 »

jcranmer wrote: post #129
razighter777 wrote: post #65

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

I've found fear of the use of floating-point in finance to be a good litmus test for how knowledgeable people are about floating-point. Because as far as I can tell, finance people almost exclusively uses (binary) floating-point [1], whereas a lot of floating-point FUD focuses on how disastrous it is for finance. And honestly, it's a bit baffling to me why so many people seem to think that floating-point is disastrou…

The answer is accounting. In accounting you want predictability and reproducibility more than anything, and you are prepared to throw away precision on that alter.

If you're summing up the cost of items in a webshop, then you're in the domain of accounting. If the result appears to be off by a single cent because of a rounding subtlety, then you're in trouble, because even though no one should care about that single cent, it will give the appearance that you don't know what you're doing. Not to mention the trouble you could get in for computing taxes wrong.

If, on the other hand, you're doing financial forecasting or computing stock price targets, then you're not in the domain of accounting, and using floating point for money is just fine.

I'm guessing from your post that your finance people are more like the latter. I could be wrong though - accountants do tend to use Excel.

Re: Beware of Fast-Math

#149

Post by dirtyhippiefree »

I’m stunned by the following admission: “If fast-math was to give always the correct results, it wouldn’t be fast-math”

If it’s not always correct, whoever chooses to use it chooses to allow error…

Sounds worse than worthless to me.

Re: Beware of Fast-Math

#150

Post by mg794613 »

Haha, the neverending cycle.

Stop trying. Let their story unfold. Let the pain commence.

Wait 30 years and see them being frustrated trying to tell the next generation.

Post reply on HN