Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of Fast-Math

simonbyrne.github.io

Post reply on HN
Print view View original

191–200 of 233 posts

Re: Beware of Fast-Math

#191

Post by adgjlsfhk1 »

cstrahan wrote: post #187
adgjlsfhk1 wrote: post #110

Earlier quoted context omitted.

no. Float64 has 16 digits of precision. Therefore even if you're dealing with trillions of dollars, you have accuracy down to the thousandth of a cent.

You might want to re-study this topic. The decimal number 0.1 has an infinitely repeating binary fraction. Consider how 1/3 in decimal is 0.33333… If you truncate that to some finite prefix, you no longer have 1/3. Now let’s suppose we know, in some context, that we’ll only ever have a finite number of digits — let’s say 5 digits after the decimal point. Then, if someone asks “what fraction is equivalent to 0.33333?”…

> you’ll likely be surprised at what you find.

I very much doubt it. My day job is writing symbolic-numeric code. The result of 0.1+0.1+0.1 != 0.3, but for rounding to bring it up to 0.31 (i.e. rounding causing an error of 1 cent), you would need to accumulate at least .005 error, which will not happen unless you lose 13 out of your 16 digits of precision, which will not happen unless you do something incredibly stupid.

Re: Beware of Fast-Math

#192

Post by adgjlsfhk1 »

layer8 wrote: post #97
eapriv wrote: post #77

Earlier quoted context omitted.

Why is it not commutative?

It actually is commutative according to IEEE-754, except that in the case of a NaN result you might get a different NaN representation.

having multiple NaNs and no spec for how they should behave feels like such an unforced error to me

Re: Beware of Fast-Math

#193

Post by Sharlin »

Measter wrote: post #183
Sharlin wrote: post #27

Earlier quoted context omitted.

Rust has some precedence for adding convenience newtypes with overloaded operators (eg. `Wrapping ´ for `I.wrapping_add(I)` etc). Such a wrapper isn't currently proposed AFAIK but there's no reason one couldn't be added in the future I believe.

For giggles, here's one I whipped up, along with an example use: https://godbolt.org/z/Eezj35dzc

Wow, that's some hardcore unrolling.

Re: Beware of Fast-Math

#194

Post by anthk »

01HNNWZ0MV43FF wrote: post #185
gamescr wrote: post #137

Earlier quoted context omitted.

I work on game engines and the problem with floats isn't on small values like 10.01 but on large ones like 400,010.01 that's when the precision wildly varies.

Lua is telling me 0.1 + 0.1 == 0.2, but 0.1 + 0.2 != 0.3. That's 64-bit precision. The issue is not with precision, but with 1/10th being a repeating decimal in binary.

Not an issue on Scheme and Common Lisp and even Forth operating directly with rationals with custom words.

Re: Beware of Fast-Math

#195

Post by MobiusHorizons »

osigurdson wrote: post #171
MobiusHorizons wrote: post #165

Earlier quoted context omitted.

the C# decimal type is not fixed point, its a floating point implementation, but just uses a base 10 exponent instead of a base 2 one like IEE754 floats. Fixed point is a general technique that is commonly done with machine integers when the necessary precision is known at compile time. It is frequently used on embedded devices that don't have a floating point unit to avoid slow software based floating point implemen…

Good to know. In a scientific domain so haven't used it previously.

Fun fact there is a decimal type on some hardware. I believe Power PC, and presumably mainframes. You can actually use it from C although it’s a software implementation on most hardware. IEEE754-2008 if you are curious.

Re: Beware of Fast-Math

#196

Post by MobiusHorizons »

jjmarr wrote: post #163
MobiusHorizons wrote: post #105

Earlier quoted context omitted.

This is what’s called a fixed point decimal type. If you need variable precision, then a decimal type might be a good idea, but fixed point removes a lot of potential foot guns if the constraints work for you.

IEEE754 defines a floating point decimal type. What are your opinions on that?

It’s very cool, but not present on most hardware. Fixed point is a lot simpler though if you are dealing with something with inherent granularity like currency

Re: Beware of Fast-Math

#197

Post by zzo38computer »

wffurr wrote: post #35
rlpb wrote: post #9

> I mean, the whole point of fast-math is trading off speed with correctness. If fast-math was to give always the correct results, it wouldn’t be fast-math, it would be the standard way of doing math. A similar warning applies to -O3. If an optimization in -O3 were to reliably always give better results, it wouldn't be in -O3; it'd be in -O2. So blindly compiling with -O3 also doesn't seem like a great idea.

If the answer can be wrong, you can make it as fast as you want.

It depend what kind of wrong answers are acceptable (and in what circumstances).

Re: Beware of Fast-Math

#198

Post by DavidVoid »

smcameron wrote: post #82

One thing I did not see mentioned in the article, or in these comments (according to ctrl-f anyway) is the use of feenableexcept()[1] to track down the source of NaNs in your code. feenableexcept(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW); will cause your code to get a SIGFPE whenever a NaN crawls out from under a rock. Of course it doesn't work with fast-math enabled, but if you're unknowingly getting NaNs without fas…

Yeah it's pretty useful to enable every once in a while just to see if anything complains.

Be very careful with it in production code though [1]. If you're in a dll then changing the FPU exception flags is a big no-no (unless you're really really careful to restore them when your code goes out of scope).

[1]: https://randomascii.wordpress.com/2016/09/16/everything-old-...

Re: Beware of Fast-Math

#199

Post by usefulcat »

CamperBob2 wrote: post #169
usefulcat wrote: post #167

Earlier quoted context omitted.

It matters if the strategy is designed to do very different things depending on whether or not the offers are locked (when bid == ask, or spread is less than 0.01). In this example, I’m talking about securities that are priced in whole cents. If you represent prices as floats, then it’s possible that the spread appears to be less (or greater) than 0.01 when it’s actually not, due to the inability of floats to exactly…

But I'm still not understanding the real-world consequences. What will those be, exactly? Any good examples or case studies to look at?

Many trading strategies operate on very thin margins. Most of the time it's less than one cent per share, often as little as a tenth of a cent per share or less.

A different example: let's say that you're trying to buy some security, and you've determined that the maximum price you can pay and still be profitable is 10.01. If you mistakenly use an order price of 10.00, you'll probably get fewer shares than you wanted, possibly none. If you mistakenly use a price of 10.02, you may end up paying too much and then that trade ends up not being profitable. If you use a price of 10.0199999 (assuming it's even possible to represent such a price via whatever protocol you're using), either your broker or the exchange will likely reject the order for having an invalid price.

Re: Beware of Fast-Math

#200

Post by DavidVoid »

cycomanic wrote: post #37

I think this article overstates the importance of the problems even for scientific software. In the scientific code I've written, noise processes are often orders of magnitude larger than what what is discussed here and I believe this applies to many (most?) simulations modelling the real world (i.e. Physics chemistry,..). At the same time enabling fast-math has often yielded a very significant (>10%) performance boo…

It matters for reproducibility between software versions, right?

I work in audio software and we have some comparison tests that compare the audio output of a chain of audio effects with a previous result. If we make some small refactoring of the code and the compiler decides to re-organize the arithmetic operations then we might suddenly get a slightly different output. So of course we disable fast-math.

One thing we do enable though, is flushing denormals to zero. That is predictable behavior and it saves some execution time.

Post reply on HN