1+1 is 2. Quick math
Optimizations Enabled by -ffast-Math
91–100 of 117 posts
Re: Optimizations Enabled by -ffast-Math
#92I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…
The whole rationale for the standard for floating-point operations is to specify the FP operations with such properties that a naive programmer will be able to write programs which will behave as expected. If you choose any option that is not compliant with the standard, that means, exactly as you have noticed, that you claim that you are an expert in FP computations and you know how to write FP programs that will gi…
Is it adviced to detect overflow and do something about it after an FP computation rather than adding pre-checks to try to avert the overflow?
Re: Optimizations Enabled by -ffast-Math
#93I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…
The whole rationale for the standard for floating-point operations is to specify the FP operations with such properties that a naive programmer will be able to write programs which will behave as expected. If you choose any option that is not compliant with the standard, that means, exactly as you have noticed, that you claim that you are an expert in FP computations and you know how to write FP programs that will gi…
Rename to: -fdisable-math-safeties
Re: Optimizations Enabled by -ffast-Math
#94Earlier quoted context omitted.
At least GCC is I believe fairly buggy in that regard since isnan is optimised out, thus you cannot check if say an incoming value read from the network is a NaN. There's some lengthy discussion about it on the big tracker iirc..
I was misremembering, was thinking about this recent convo on LLVM mailinglist: https://lists.llvm.org/pipermail/llvm-dev/2021-September/152...
I don't think you are misremembering; -ffast-math causes __builtin_isnan and __builtin_isinf to unconditionally return zero, for both gcc and clang:
https://godbolt.org/z/1qjxvW3KK
I found this out the hard way several years ago and still have my own implementations of these functions laying about in case I want to use -ffast-math.
Re: Optimizations Enabled by -ffast-Math
#95Earlier quoted context omitted.
The whole rationale for the standard for floating-point operations is to specify the FP operations with such properties that a naive programmer will be able to write programs which will behave as expected. If you choose any option that is not compliant with the standard, that means, exactly as you have noticed, that you claim that you are an expert in FP computations and you know how to write FP programs that will gi…
Is it adviced to detect overflow and do something about it after an FP computation rather than adding pre-checks to try to avert the overflow?
So it depends on how you want to detect overflow.
Trapping on math exceptions or otherwise pulling an emergency break doesn't really fit with modern hardware design if you want performance (for one thing, what do you do with the rest of your vector? do you just live with a pipeline flush?).
Adding targeted checks used sparingly can work, but probably requires a deeper understanding of the underlying numerical analysis of your problem. Generally just checking for NaN or Inf at the end is a better solution as these are in most cases absorbing states.
Re: Optimizations Enabled by -ffast-Math
#96Earlier quoted context omitted.
> When the input is one of those points directly, That's against one of the rules of well-behaved floating point programming: Never test for equality.
It's worth being a bit more explicit about this. The appearance of NaNs doesn't result from breaking the "never test for equality" rule. It happens when the point where you're evaluating the function just happens to exactly match one of the interpolation points. But if you decide to deal with the NaN issue by (1) testing whether your input exactly matches any of the interpolation points, or (2) testing the result for…
> perhaps less-obvious bad stuff happens when x almost exactly equals one of the x_j, and an equality test won't catch that.
So, does it? Actually, I think not.
You are correct, it does not. Quite the opposite: the results are excellent when x doesn’t quite equal one of the interpolation nodes, because you end up with a value of S · y / S, where S = «very big but not close to overflowing float» and y = «expected output value».
The only potentially problematic case is if S · y overflows but S does not. But floating point numbers have a ton of dynamic range, so unless you are looking at points really, really close to zero, with very large outputs (say, x − x_j = 1e-200, y = 1e+200), there is no problem. And the failure case there is that you end up with Inf instead of 1e200 as an output.
It isn’t a problem in practice, and if you are worried about it as a problem in theory, you can take care in choice of interpolation nodes or do some scaling of the output values.
Re: Optimizations Enabled by -ffast-Math
#97Earlier quoted context omitted.
This depends how you define “gone wrong”. In evaluating rational functions (a useful tool for approximating all sorts of other functions), one efficient and well behaved algorithm is the “barycentric formula”, based on interpolating function values at various specific input points. When the input is one of those points directly, this formula results in Inf / Inf = NaN. Evaluation code needs to check for this case, an…
What is the rationale for liking rational functions? Years ago i had some interpolation to do, and used the barycentric rational from Boost [1]. The resulting curve was all over the place. A natural cubic spline gave a much more subjectively sensible-looking fit. [1] https://www.boost.org/doc/libs/1_77_0/libs/math/doc/html/mat...
Without knowing a lot more about the context and what precisely you did, it’s impossible to comment about your negative experience.
> What is the rationale for liking rational functions?
They are cheap to evaluate, (relatively) easy to prove things about symbolically, and will efficiently approximate any reasonably smooth function.
Re: Optimizations Enabled by -ffast-Math
#98Earlier quoted context omitted.
Hm, interesting. I also tried some numerical experiments to sanity-check and I got reasonable results right up to 1ulp away from the interpolation points. I tried again using exactly your example -- f(x) = 10+10x, interpolation points at 0 and 1 -- and again everything seemed fine. I wonder what we're doing differently, or whether we happened to test at different points. Here's my code: def interpolate1(f,x,xs,ws): p…
#include #include double interp(double x, double x1, double y1, double x2, double y2) { double node_poly = (x - x1) * (x - x2); double w1 = 1.0 / (x1 - x2); double w2 = 1.0 / (x2 - x1); double y = node_poly * ((w1 * y1) / (x - x1) + (w2 * y2) / (x - x2)); return y; } int main() { for (int p = -300; p >= -330; p -= 1) { double x = pow(10, p); double y = interp(x, 0, 10, 1, 20); printf("x = %.6g, y = %.6g\n", x, y); }…
You implemented the wrong formula. You don’t want to explicitly calculate "node poly".
The formula you want is the “second barycentric formula”, (4.2) at the top of page 505 of https://people.maths.ox.ac.uk/trefethen/barycentric.pdf
Or formula (2.6) halfway down page 2 of https://people.maths.ox.ac.uk/trefethen/publication/PDF/2011...
Also cf. Higham (2004) “The numerical stability of barycentric Lagrange interpolation” https://www.maths.manchester.ac.uk/~higham/narep/narep440.pd...
If you want to implement the “first barycentric formula” instead you need to take some additional care to avoid overflow/underflow.
As the papers above note, the choice of interpolation nodes makes a big difference. If you have an arbitrary function to approximate on an arbitrary complex domain, take a look at https://arxiv.org/abs/1612.00337
Re: Optimizations Enabled by -ffast-Math
#99Earlier quoted context omitted.
Can you think of a function where the input is valid, the output is NaN and nothing has gone wrong in the process? I can't think of any, haven't experienced any, not heard of any examples of it, so you're welcome to break my ignorance on the subject.
Calculating the average value for some observation in a time bucket, where some buckets may have no observation (resulting in 0/0=nan), finally taking some kind of summary of these ignoring nan values (there is a whole library of functions for this in numpy for example). NaN and Inf are extremely valuable and useful encodings to represent real things. Particularly signed infinity is wonderful, as functions like exp b…
>Calculating the average value for some observation in a time bucket, where some buckets may have no observation (resulting in 0/0=nan)
But there are many ways such a calculation could result in NaN besides a division. For example, intermediate sums could result in something like inf - inf. I find it rare that you can definitely conclude that a NaN means some specific thing.
Re: Optimizations Enabled by -ffast-Math
#100Why is this compiler optimization beneficial with -ffinite-math-only and -fno-signed-integers?
From
if (x > y) {
do_something();
} else {
do_something_else();
}
if (x