Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizations Enabled by -ffast-Math

kristerw.github.io

Post reply on HN
Print view View original

61–70 of 117 posts

Re: Optimizations Enabled by -ffast-Math

#61

Post by jlokier »

jacobolus wrote: post #53
pinteresting wrote: post #43

Earlier quoted context omitted.

Can you think of a function where the input is valid, the output is NaN and nothing has gone wrong in the process? I can't think of any, haven't experienced any, not heard of any examples of it, so you're welcome to break my ignorance on the subject.

This depends how you define “gone wrong”. In evaluating rational functions (a useful tool for approximating all sorts of other functions), one efficient and well behaved algorithm is the “barycentric formula”, based on interpolating function values at various specific input points. When the input is one of those points directly, this formula results in Inf / Inf = NaN. Evaluation code needs to check for this case, an…

> When the input is one of those points directly,

That's against one of the rules of well-behaved floating point programming: Never test for equality.

Re: Optimizations Enabled by -ffast-Math

#62

Post by hurrrrrrrr »

gpderetta wrote: post #25
rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

I think the documentation is fairly clear: "Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs." The obvious implication is that is that arguments and results are always finite and as a programmer you are responsible of guaranteeing the correct preconditions. Certainly do not use finite-math-only when dealing with external data.

>Certainly do not use finite-math-only when dealing with external data.

so... never?

Re: Optimizations Enabled by -ffast-Math

#63

Post by dkersten »

ChrisLomont wrote: post #57
pinteresting wrote: post #43

Earlier quoted context omitted.

Can you think of a function where the input is valid, the output is NaN and nothing has gone wrong in the process? I can't think of any, haven't experienced any, not heard of any examples of it, so you're welcome to break my ignorance on the subject.

There are algorithms that behave correctly but require intermediate Inf or NaN to work. If I recall there is one related to finding bounding box intersections in ray tracing. I know when I introduce such code into a codebase I put a large "DO NOT CHANGE THIS..." style comment with an explanation so some enterprising programmer doesn't put some if statements in to remove what they think is an error, when it is not. I'…

Sure, of course they have uses otherwise why would they exist. But I would argue that the examples you have aren’t the common case — most people’s floating point calculations do not rely on these special values and seeing them denotes something went wrong.

Re: Optimizations Enabled by -ffast-Math

#64

Post by stncls »

pinteresting wrote: post #29
rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

Generally if you're seeing a NaN/Inf something has gone wrong, It's very difficult to gracefully recover from and if you tried I think you would lose both sanity and performance! Regarding performance, the cost of a real division is about 3-4 orders worse performance than an if statement that is very consistent, but the usual way is to have fast/safe versions of functions, where you need performance and can deduce if…

An example use of Inf is bounds on a value. Say you want to express that x sometimes has upper and/or lower bounds. With Inf you can simply write

    l
accepting that l is sometimes -Inf and u is sometimes +Inf. Without Inf, you get four different cases to handle. This is particularly handy when operations get slightly more complex, like transforming

    l'
into the canonical form above, for some finite b. With Inf you can simply write

    l = l' - b
    u = u' - b
and things will work as one expects. Again, without Inf, multiple cases to handle correctly.

Re: Optimizations Enabled by -ffast-Math

#65

Post by adrian_b »

jlokier wrote: post #61
jacobolus wrote: post #53

Earlier quoted context omitted.

This depends how you define “gone wrong”. In evaluating rational functions (a useful tool for approximating all sorts of other functions), one efficient and well behaved algorithm is the “barycentric formula”, based on interpolating function values at various specific input points. When the input is one of those points directly, this formula results in Inf / Inf = NaN. Evaluation code needs to check for this case, an…

> When the input is one of those points directly, That's against one of the rules of well-behaved floating point programming: Never test for equality.

Like any rule, this has exceptions.

You should never test for equality numbers which are known only approximately, which is the case for most FP numbers.

There are nonetheless cases when certain FP values are known exactly and it is OK to test them for equality.

In general the rule does not depend on number representation, it applies equally to floating-point numbers, fixed-point numbers, rational numbers and even large integers in some cases.

What counts is whether a number is known exactly or only approximately.

Re: Optimizations Enabled by -ffast-Math

#66

Post by adrian_b »

vanviegen wrote: post #59
ImprobableTruth wrote: post #46

Earlier quoted context omitted.

Do you count under/overflow resulting in Inf as 'something has gone wrong'? If so, why would gracefully recovering be hard?

Because you're about 15 stack frames deep into some calculation when the problem gets noticed. So how do you handle this? You could throw an exception (or similar), which would allow the UI to state something semi-helpful like 'Calculation failed'. But recover? Usually not. I think NaN and inf can be really useful in specific cases, but they probably shouldn't be allowed to propagate through your program, and shouldn…

When you do not want such values to be propagated, it is always possible to choose to have overflow exceptions instead of infinities and undefined operation exceptions instead of NaNs. Also underflow exceptions instead of denormals.

The programmer should choose which is most appropriate for an application. Choosing to both ignore the exceptions and not generate a special value that can be examined later, that is seldom acceptable.

Re: Optimizations Enabled by -ffast-Math

#67

Post by 0xTJ »

rocqua wrote: post #11

I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…

That's the way a number of those special flags work. They are a contract to the compiler saying "act like this can't happen, I promise it won't".

The picking and choosing specific flags enable might be better in general. The flag `-funsafe-math-optimizations` seems to be useful as general-purpose, although as this article mentions, some of those optimizations need the more unsafe ones to become active.

I would say that `-ffast-math` is a dangerously safe-sounding flag, `-funsafe-fast-math` would be better IMO.

Re: Optimizations Enabled by -ffast-Math

#68

Post by adrian_b »

colejohnson66 wrote: post #30
nly wrote: post #19

Earlier quoted context omitted.

He means simple stuff like ensuring a function like float unit_price (float volume, int count) { return volume / count; } is only called where count > 0

But couldn’t the compiler optimize out the check because you’re guaranteeing to never divide by 0, so logically count would never be 0. Akin to undefined behavior almost.

What the compiler assumes depends on the compilation options.

Unfortunately most compilers have too permissive default options.

C and C++ programs should always be compiled, at least during development, with strict options, e.g.:

-fsanitize=undefined -fsanitize-undefined-trap-on-error

or equivalent options, which eliminate all the horror stories with unexpected behavior caused by integer divide-by-zero or overflow, pointer overflow, out-of-bounds access and many others.

Because unlike integer operations the FP operations are standardized, there is no need for compiler options, the behavior for exceptions can be controlled from the program with the functions defined in in C or in C++.

Re: Optimizations Enabled by -ffast-Math

#69

Post by ChrisLomont »

dkersten wrote: post #63
ChrisLomont wrote: post #57

Earlier quoted context omitted.

There are algorithms that behave correctly but require intermediate Inf or NaN to work. If I recall there is one related to finding bounding box intersections in ray tracing. I know when I introduce such code into a codebase I put a large "DO NOT CHANGE THIS..." style comment with an explanation so some enterprising programmer doesn't put some if statements in to remove what they think is an error, when it is not. I'…

Sure, of course they have uses otherwise why would they exist. But I would argue that the examples you have aren’t the common case — most people’s floating point calculations do not rely on these special values and seeing them denotes something went wrong.

Of course they're not the common case, but the parent wanted examples where they are used in calculations that do have nice answers.

So they are useful. They are more useful than most people realize that simply assume they only mark errors.

And without them in your case, you would not be alerted that a calculation went wrong. So even there they are very useful. Not using them correctly is like ignoring file API errors in your programs - sure they are rare, but you need to understand and handle them if you want to make good software.

Re: Optimizations Enabled by -ffast-Math

#70

Post by dkersten »

ChrisLomont wrote: post #69
dkersten wrote: post #63

Earlier quoted context omitted.

Sure, of course they have uses otherwise why would they exist. But I would argue that the examples you have aren’t the common case — most people’s floating point calculations do not rely on these special values and seeing them denotes something went wrong.

Of course they're not the common case, but the parent wanted examples where they are used in calculations that do have nice answers. So they are useful. They are more useful than most people realize that simply assume they only mark errors. And without them in your case, you would not be alerted that a calculation went wrong. So even there they are very useful. Not using them correctly is like ignoring file API error…

Ah right, fair enough! They are, indeed, good examples of the usefulness.
Post reply on HN