Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizations Enabled by -ffast-Math

kristerw.github.io

Post reply on HN
Print view View original

21–30 of 117 posts

Re: Optimizations Enabled by -ffast-Math

#21

Post by marcan_42 »

adrian_b wrote: post #7
ptidhomme wrote: post #5

Do some applications actually use denormal floats ? I'm curious.

Denormal floats are not a purpose, they are not used intentionally. When the CPU generates denormal floats on underflow, that ensures that underflow does not matter, because the errors remain the same as at any other floating-point operation. Without denormal floats, underflow must be an exception condition that must be handled somehow by the program, because otherwise the computation errors can be much higher than e…

Flushing denormals to zero only matters if your calculations are already running into the lower end of floating-point exponents (and even with denormals, if they're doing that, they're going to run into lost precision anyway sooner or later).

The useful thing denormals do is make the loss of precision at that point gradual, instead of sudden. But you're still losing precision, and a few orders of magnitude later you're going to wind up at 0 either way.

If your formulas are producing intermediate results with values that run into the lower end of FP range, and it matters that those values retain precision there, then you're either using the wrong FP type or you're using the wrong formulas. Your code is likely already broken, the breakage is just rarer than in flush-to-zero mode.

So just enable FTZ mode, and if you run into issues, you need to fix that code (e.g. don't divide by values that can be too close to 0), not switch denormals on.

Re: Optimizations Enabled by -ffast-Math

#22

Post by marcan_42 »

ptidhomme wrote: post #18
adrian_b wrote: post #7

Earlier quoted context omitted.

Denormal floats are not a purpose, they are not used intentionally. When the CPU generates denormal floats on underflow, that ensures that underflow does not matter, because the errors remain the same as at any other floating-point operation. Without denormal floats, underflow must be an exception condition that must be handled somehow by the program, because otherwise the computation errors can be much higher than e…

Thanks for this explanation. But yeah, I meant : are there applications where use of denormals vs. flush-to-zero is actually useful... ? If your variable will be nearing zero, and e.g. you use it as a divider, don't you need to handle a special case anyway ? Just like you should handle your integer overflows. I'm seeing denormals as an extension of the floating point range, but with a tradeoff that's not worth it. Ma…

I agree with you; if your code "works" with denormals and "doesn't work" with flush-to-zero, then it's already broken and likely doesn't work for some inputs anyway, you just haven't run into it yet.

Re: Optimizations Enabled by -ffast-Math

#25

Post by gpderetta »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

I think the documentation is fairly clear:

"Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs."

The obvious implication is that is that arguments and results are always finite and as a programmer you are responsible of guaranteeing the correct preconditions.

Certainly do not use finite-math-only when dealing with external data.

Re: Optimizations Enabled by -ffast-Math

#26

Post by gpderetta »

mhh__ wrote: post #23

If you use the LLVM D compiler you can opt in or out of these individually on a per-function basis. I don't trust them globally.

with gcc you can also use #pragma gcc optimize or __attribute__(optimize(...)) for a similar effect.

It is not 100% bug free (at least it didn't use to) and often it prevents inlining a function into another having different optimization levels (so in practice its use has to be coarse grained).

Re: Optimizations Enabled by -ffast-Math

#27

Post by contravariant »

okl wrote: post #12
rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.

And don't do any division.

Easy.

Re: Optimizations Enabled by -ffast-Math

#28

Post by stabbles »

mhh__ wrote: post #23

If you use the LLVM D compiler you can opt in or out of these individually on a per-function basis. I don't trust them globally.

In Julia you can do `@fastmath ...`, which is basically just a find-and-replace of math operations and does not propagate into functions called in that code block, even when they are ultimately inlined. So what it does is:

    julia> @macroexpand @fastmath x + y
    :(Base.FastMath.add_fast(x, y))
And maybe that's a good thing, because the scope of @fastmath is as limited as it gets.

Re: Optimizations Enabled by -ffast-Math

#29

Post by pinteresting »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

Generally if you're seeing a NaN/Inf something has gone wrong, It's very difficult to gracefully recover from and if you tried I think you would lose both sanity and performance!

Regarding performance, the cost of a real division is about 3-4 orders worse performance than an if statement that is very consistent, but the usual way is to have fast/safe versions of functions, where you need performance and can deduce if something is always/never true, create/use the fast function but by default everything uses the slower/safer function.

Re: Optimizations Enabled by -ffast-Math

#30

Post by colejohnson66 »

nly wrote: post #19
orangepanda wrote: post #17

Earlier quoted context omitted.

As the compiler assumes you wont produce such values, wouldnt it also optimise those range checks away?

He means simple stuff like ensuring a function like float unit_price (float volume, int count) { return volume / count; } is only called where count > 0

But couldn’t the compiler optimize out the check because you’re guaranteeing to never divide by 0, so logically count would never be 0. Akin to undefined behavior almost.
Post reply on HN