Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizations Enabled by -ffast-Math

kristerw.github.io

Post reply on HN
Print view View original

11–20 of 117 posts

Re: Optimizations Enabled by -ffast-Math

#11

Post by rocqua »

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying:

The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used.

I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So optimize things that break the standard".

But instead it seems like this also implies a promise to the compiler. A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude.

This changes the flag from saying "don't care about standards, make it fast" to "I hereby guarantee this code meets a stricter standard" where also it becomes quite hard to actually deduce you will meet this standard. Especially if you want to actually keep your performance. Because if you need to start putting all divisions in if statements to prevent getting Inf or NaN, that is a huge performance penalty.

Re: Optimizations Enabled by -ffast-Math

#12

Post by okl »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude.

Cumbersome but not that difficult: Range-check all input values.

Re: Optimizations Enabled by -ffast-Math

#13

Post by mjburgess »

andraz wrote: post #10
atoav wrote: post #8

Earlier quoted context omitted.

Also maybe you woild like to have more granular control over which parts of your program has the priority on speed and which part favours accuracy. Maybe this could be done with a seperate type (e.g. ff64) or a decorator (which would be useful if you want to enable this for someone elses library).

Maybe... or maybe there would be just a flag to enable all this and be fine with consequences.

..have you used Rust?

> just a flag to enable all this and be fine with consequences

Is probably the opposite of what the language is about.

Re: Optimizations Enabled by -ffast-Math

#15

Post by adrian_b »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

The whole rationale for the standard for floating-point operations is to specify the FP operations with such properties that a naive programmer will be able to write programs which will behave as expected.

If you choose any option that is not compliant with the standard, that means, exactly as you have noticed, that you claim that you are an expert in FP computations and you know how to write FP programs that will give the desired results even with FP operations that can behave in strange ways.

So yes, that means that you become responsible to either guarantee that erroneous results do not matter or that you will take care to always check the ranges of input operands, as "okl" has already posted, to ensure that no overflows, underflows or undefined operations will happen.

Re: Optimizations Enabled by -ffast-Math

#16

Post by praptak »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

Not only math optimizations are like this. Strict aliasing is a promise to the compiler to not make two pointers of different types point to the same address.

You technically make that promise by writing in C (except a narrow set of allowed conversions) but most compilers on standard settings do let you get away with it.

Re: Optimizations Enabled by -ffast-Math

#17

Post by orangepanda »

okl wrote: post #12
rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.

As the compiler assumes you wont produce such values, wouldnt it also optimise those range checks away?

Re: Optimizations Enabled by -ffast-Math

#18

Post by ptidhomme »

adrian_b wrote: post #7
ptidhomme wrote: post #5

Do some applications actually use denormal floats ? I'm curious.

Denormal floats are not a purpose, they are not used intentionally. When the CPU generates denormal floats on underflow, that ensures that underflow does not matter, because the errors remain the same as at any other floating-point operation. Without denormal floats, underflow must be an exception condition that must be handled somehow by the program, because otherwise the computation errors can be much higher than e…

Thanks for this explanation. But yeah, I meant : are there applications where use of denormals vs. flush-to-zero is actually useful... ? If your variable will be nearing zero, and e.g. you use it as a divider, don't you need to handle a special case anyway ? Just like you should handle your integer overflows.

I'm seeing denormals as an extension of the floating point range, but with a tradeoff that's not worth it. Maybe I got it wrong ?

Re: Optimizations Enabled by -ffast-Math

#19

Post by nly »

orangepanda wrote: post #17
okl wrote: post #12

Earlier quoted context omitted.

> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.

As the compiler assumes you wont produce such values, wouldnt it also optimise those range checks away?

He means simple stuff like ensuring a function like

    float
    unit_price (float volume, int count) {
       return volume / count;
    }
is only called where count > 0

Re: Optimizations Enabled by -ffast-Math

#20

Post by jcelerier »

orangepanda wrote: post #17
okl wrote: post #12

Earlier quoted context omitted.

> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.

As the compiler assumes you wont produce such values, wouldnt it also optimise those range checks away?

At least GCC is I believe fairly buggy in that regard since isnan is optimised out, thus you cannot check if say an incoming value read from the network is a NaN. There's some lengthy discussion about it on the big tracker iirc..
Post reply on HN