Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of fast-math

simonbyrne.github.io

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Beware of fast-math

#51

Post by willis936 »

adgjlsfhk1 wrote: post #48
willis936 wrote: post #46

I use single precision floating point to save memory and computation in applications where it makes sense. I had a case where I didn't care about the vertical precision of a signal very much. It had a sample rate in the tens of thousands of samples per second. I was generating a sinusoid and transmitting it. On the receiver the signal would become garbled after about a minute. I slapped my head and immediately realiz…

IMO, sinpi(x)=sin(pi*x) is a better function because it does much better here. the regular trig functions are approximately 20% slower for most implementations in order to accurately reduce mod 2pi, while reducing mod 2 is pretty much trivial.

I think the real solution I should have adopted is incrementing t like this:

  t = mod(t + ts, 1 / f)
Since I'm just sending a static frequency the range of time never needs to be beyond one period. However, using a double here is far from the critical path in increasing performance and it all runs fast enough anyway.

Re: Beware of fast-math

#53

Post by simonbyrne »

smitop wrote: post #52

The LLVM IR is more expressive than clang is for expressing fast-math: it supports making an operation use fast-math optimization on a per operation basis ( https://llvm.org/docs/LangRef.html#fastmath ).

Do you know what happens when you have ops with different flags? e.g. if you have (a + b) + c, where one + allows reassoc but one doesn't?

Re: Beware of fast-math

#54

Post by Quekid5 »

pavpanchekha wrote: post #27

Fun fact: when working on Herbie ( http://herbie.uwplse.org ), our automated tool for reducing floating-point error by rearranging your mathematical expressions, we found that fast-math often undid Herbie's improvements. In a sense, Herbie and fast-math are opposites: one makes code more accurate (sometimes slower, sometimes faster), while the other makes code faster (sometimes less accurate, sometimes more).

That's a real "fun fact" kind of thing. Love it.

Of course, the 'real' solution is actual Numerical Analysis (as I'm sure you know) to keep the error properly bounded, but it's really interesting to have a sort of middle ground which just stabilizes the math locally... which might be good enough.

Other fun fact: Numerical Analysis is a thing that's really hard to imagine unless you happen to be introduced to it during an education. It's so obviously a thing once you've heard of it, but REALLY hard to come up with ex nihilo.

Re: Beware of fast-math

#55

Post by djmips »

simonbyrne wrote: post #15
dmitrygr wrote: post #3

Contrarian waypoint: beware of not-fast-math. Making things like atan2f and sqrtf set errno takes you down a very slow path, costing you significant perf in cases where you likely do not want it. And most math will work fine with fast-math, if you are careful how you write it. (Free online numerical methods classes are available, eg [1]) Without fast-math most compilers cannot even use FMA instructions (costing you u…

My point isn't that fast-math isn't useful: it very much is. The problem is that it is a whole grab bag of things that can do very dangerous things. Rather than using a sledgehammer, you should try to be selective and enable only the useful optimizations, e.g. you could just enable -ffp-contract=fast and -fno-math-errno.

One thing I don't think you pointed out is that tracking down issues with NaNs seems hard with fast-math since, I believe, it also disables any exceptions that might be useful to being alerted to their formation?

Re: Beware of fast-math

#56

Post by pcwalton »

Joker_vD wrote: post #20
StefanKarpinski wrote: post #11

Earlier quoted context omitted.

That's true, but the danger of flushing subnormals to zero is correspondingly worse because it's global CPU state and there's commonly used code that relies on not flushing subnormals to zero in order to work correctly, like `libm`. The example linked in the post is of a case where loading a shared library that had been compiled with `-Ofast` (which includes `-ffast-math`) broke a completely unrelated package because…

Wait, what is "local" CPU state/hardware flag? In any case, since x64 ABI doesn't require MXCSR to be in any particular state on function entry, libm should set/clear whatever control flags it needs on its own (and restore them on exit since MXCSR control bits are defined to be callee-saved).

That wouldn't be practical. C math library performance really matters for numerical-intensive apps like games.

Re: Beware of fast-math

#58

Post by bruce343434 »

jjgreen wrote: post #57

One trick that I happened upon was speeding up complex multiplication (like a factor of 5) under gcc with the --enable-cx-fortran switch.

-fcx-fortran-rules

    Complex multiplication and division follow Fortran rules. Range reduction is done as part of complex division, but there is no checking whether the result of a complex multiplication or division is NaN + I*NaN, with an attempt to rescue the situation in that case.

    The default is -fno-cx-fortran-rules.
[1] https://gcc.gnu.org/onlinedocs/gcc-9.3.0/gcc/Optimize-Option...

Re: Beware of fast-math

#60

Post by mhh__ »

physicsguy wrote: post #42
mhh__ wrote: post #26

Earlier quoted context omitted.

D (LDC) lets you control the flags on a per function basis. So one can (and we do at work) have @optmath which is a specific set of flags (just a value we defined at compile time, the compiler recognizes it as a UDA) we want as opposed to letting the compiler bulldoze everything.

You can do that on a per-compiler basis for e.g. with #pragma GCC optimize(“fast-math")

Hopefully it still works as an attribute but my point is that you can (say) opt in to allowing more liberal use of FMA without (say) opting in to aggressive NaN assumptions
Post reply on HN