I wish the Twitter links in this article weren't broken.
Change X to xcancel
Beware of Fast-Math
11–20 of 233 posts
Re: Beware of Fast-Math
#12I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…
Does that mean that a physics engine written with these operations will always compile to yield the same deterministic outcomes across different platforms (assuming they correctly implement (or able to do so) algebraic operations)?
Re: Beware of Fast-Math
#13(2021)
Previous discussion: Beware of fast-math (Nov 12, 2021, https://news.ycombinator.com/item?id=29201473)
Re: Beware of Fast-Math
#14I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…
Does that mean that a physics engine written with these operations will always compile to yield the same deterministic outcomes across different platforms (assuming they correctly implement (or able to do so) algebraic operations)?
No, there is no guarantee which (if any) optimizations are applied, only that they may be applied. For example a fused multiply-add instruction may be emitted for a*b + c on platforms which support it, which is not cross-platform.
Re: Beware of Fast-Math
#15> I mean, the whole point of fast-math is trading off speed with correctness. If fast-math was to give always the correct results, it wouldn’t be fast-math, it would be the standard way of doing math. A similar warning applies to -O3. If an optimization in -O3 were to reliably always give better results, it wouldn't be in -O3; it'd be in -O2. So blindly compiling with -O3 also doesn't seem like a great idea.
The optimisations in -O3 aren't supposed to give incorrect results. They're not in -O2 because they make a more aggressive space/speed tradeoff or increase compile times more significantly. In the same way, the optimisations in -O2 are not meant to be less correct than -O1, but they aren't in that group for similar reasons.
-Ofast is the 'dangerous' one. (It includes -ffast-math).
Re: Beware of Fast-Math
#16I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…
Does that mean that a physics engine written with these operations will always compile to yield the same deterministic outcomes across different platforms (assuming they correctly implement (or able to do so) algebraic operations)?
No, the result may depend on how the compiler reorders them, which could be different on different platforms.
Re: Beware of Fast-Math
#17Previously discussed at https://news.ycombinator.com/item?id=29201473 (which the article itself links to at the end).
On Forth, there's the philosophy of the fixed point: https://www.forth.com/starting-forth/5-fixed-point-arithmeti... With 32 and 64 bit numbers, you can just scale decimals up. So, Torvalds was right. On dangerous contexts (uper-precise medical doses, FP has good reasons to exist, and I am not completely sure). Also, both Forth and Lisp internally suggest to use represented rationals before floating point numbers. Ev…
Those rational numbers fly out the window as soon as your math involves any kind of more complicated trigonometry, or even a square root…
Re: Beware of Fast-Math
#18I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…
Are these calls going to clear the FTZ and DAZ flags in the MXCSR on x86? And FZ & FIZ in the FPCR on ARM?
I don't believe so, no. Currently these operations only set the LLVM flags to allow reassociation, contraction, division replaced by reciprocal multiplication, and the assumption of no signed zeroes.
This can be expanded in the future as LLVM offers more flags that fall within the scope of algebraically motivated optimizations.
Re: Beware of Fast-Math
#19Earlier quoted context omitted.
Are these calls going to clear the FTZ and DAZ flags in the MXCSR on x86? And FZ & FIZ in the FPCR on ARM?
I don't believe so, no. Currently these operations only set the LLVM flags to allow reassociation, contraction, division replaced by reciprocal multiplication, and the assumption of no signed zeroes. This can be expanded in the future as LLVM offers more flags that fall within the scope of algebraically motivated optimizations.
Ah sorry I misunderstood and thought this API was for the other way around, i.e. forbidding "unsafe" operations. (I guess the question reverses to setting those flags)
('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D)
Re: Beware of Fast-Math
#20Earlier quoted context omitted.
I don't believe so, no. Currently these operations only set the LLVM flags to allow reassociation, contraction, division replaced by reciprocal multiplication, and the assumption of no signed zeroes. This can be expanded in the future as LLVM offers more flags that fall within the scope of algebraically motivated optimizations.
Ah sorry I misunderstood and thought this API was for the other way around, i.e. forbidding "unsafe" operations. (I guess the question reverses to setting those flags) ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D)
> ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D)
Okay, the floating point operations are literally algebraic (they form an algebra) but they don't follow some common algebraic properties like associativity. The linked tracking issue itself acknowledges that:
> Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.
Also this comment https://github.com/rust-lang/rust/issues/136469#issuecomment...
> > On that note I added an unresolved question for naming since algebraic isn't the most clear indicator of what is going on. > > I think it is fairly clear. The operations allow algebraically justified optimizations, as-if the arithmetic was real arithmetic. > > I don't think you're going to find a clearer name, but feel free to provide suggestions. One alternative one might consider is real_add, real_sub, etc.
Then retorted here https://github.com/rust-lang/rust/issues/136469#issuecomment...
> These names suggest that the operations are more accurate than normal, where really they are less accurate. One might misinterpret that these are infinite-precision operations (perhaps with rounding after a whole sequence of operations). > > The actual meaning isn't that these are real number operations, it's quite the opposite: they have best-effort precision with no strict guarantees. > > I find "algebraic" confusing for the same reason. > > How about approximate_add, approximate_sub?
And the next comment
> Saying "approximate" feels imperfect, as while these operations don't promise to produce the exact IEEE result on a per-operation basis, the overall result might well be more accurate algebraically. E.g.: > > (...)
So there's a discussion going on about the naming