Rust and C++ on Floating-Point Intensive Code

Post by tom_mellior » Thu, Oct 24, 2019, 1:06 PM UTC

fluffything wrote: post #35

nestorD wrote: post #29

Earlier quoted context omitted.

The one very important thing that often get destroyed by compiler using associativity is the TwoSum Error free transform which is a vital composant of several algorithms that deal with numerical error (most notably the Kahan Summation). The problem is mentionned in the Wikipedia page of the Kahan summation (and I have been able to reproduce it with gcc) : https://en.wikipedia.org/wiki/Kahan_summation_algorithm#Poss..…

I think the key issue with the optimizations that ICC is performing for C++ but Rust is not doing in this case is just FP-contraction, which is related to, but not the same as, assuming associativity. The RFC about that is https://github.com/rust-lang/rfcs/pull/2686 , where you see users kind of split into the "I want faster binaries" and "I want more deterministic execution" camps. Neither are wrong TBH. Some people…

    r+=(c[i]-a[i]*b[i])*a[i]*(c[i]-a[i]*b[i]);

    r0 += (c[i+0]-a[i+0]*b[i+0])*a[i+0]*(c[i+0]-a[i+0]*b[i+0]);
    r1 += (c[i+1]-a[i+1]*b[i+1])*a[i+1]*(c[i+1]-a[i+1]*b[i+1]);
    r2 += (c[i+2]-a[i+2]*b[i+2])*a[i+2]*(c[i+2]-a[i+2]*b[i+2]);
    ...

Post by nine_k » Thu, Oct 24, 2019, 1:08 PM UTC

diamondlovesyou wrote: post #4

I have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to igno…

Post by stefs » Thu, Oct 24, 2019, 1:20 PM UTC

fluffything wrote: post #25

cannam wrote: post #16

Earlier quoted context omitted.

> I'm also partial to exposing it per-function so the control is actually in the hands of the people writing the code that know the context As a C++ programmer who routinely uses fast-math "until something breaks" with DSP code, I would find that capability very attractive.

That's kind of at odds with Rust guarantee that your code never breaks.

Post by BubRoss » Thu, Oct 24, 2019, 1:24 PM UTC

fluffything wrote: post #26

jacobolus wrote: post #15

Earlier quoted context omitted.

Some algorithms guarantee that some arithmetic operation(s) applied to 2+ floating point inputs will result in a list of floating point outputs which when summed have exactly the correct result. This gets all screwed up if you mess with the order of operations or the rounding of intermediate results. e.g. https://www.cs.cmu.edu/~quake/robust.html Some keywords to look for: “compensated arithmetic”, “error-free transf…

Fair, I think it would be very helpful for Rust if some expert actually knows of a specific example for which this is the case. I think there is an RFC about enabling floating-point contraction by default, that would "silently" be able to do some of these transformations depending on the optimization level.

Post by steveklabnik » Thu, Oct 24, 2019, 1:39 PM UTC

stefs wrote: post #45

fluffything wrote: post #25

Earlier quoted context omitted.

That's kind of at odds with Rust guarantee that your code never breaks.

rust doesn't guarantee anything if you opt out of the guarantees. two examples that come to mind: unsafe and maybe bounds checks in release mode. fastmath is probably different anyway, as the "breaking" is on a floating point logic level, as in: results become imprecise, but not exactly "wrong" - as in undefined behaviour. but i don't know fastmath, so i might be wrong.