Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of Fast-Math

simonbyrne.github.io

Post reply on HN
Print view View original

21–30 of 233 posts

Re: Beware of Fast-Math

#21

Post by Sharlin »

evrimoztamur wrote: post #12
orlp wrote: post #4

I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…

Does that mean that a physics engine written with these operations will always compile to yield the same deterministic outcomes across different platforms (assuming they correctly implement (or able to do so) algebraic operations)?

It's more like the opposite. These tell the compiler to assume for optimization purposes that floats are associative and so on (ie. algebraic), even when in reality they aren't. So the results may vary depending on what transformations the compiler performs – in particular, they may vary between optimized and non-optimized builds, which normally isn't allowed.

Re: Beware of Fast-Math

#22

Post by rlpb »

CamouflagedKiwi wrote: post #15
rlpb wrote: post #9

> I mean, the whole point of fast-math is trading off speed with correctness. If fast-math was to give always the correct results, it wouldn’t be fast-math, it would be the standard way of doing math. A similar warning applies to -O3. If an optimization in -O3 were to reliably always give better results, it wouldn't be in -O3; it'd be in -O2. So blindly compiling with -O3 also doesn't seem like a great idea.

The optimisations in -O3 aren't supposed to give incorrect results. They're not in -O2 because they make a more aggressive space/speed tradeoff or increase compile times more significantly. In the same way, the optimisations in -O2 are not meant to be less correct than -O1, but they aren't in that group for similar reasons. -Ofast is the 'dangerous' one. (It includes -ffast-math).

> The optimisations in -O3 aren't supposed to give incorrect results.

I didn't mean to imply that they result in incorrect results.

> they make a more aggressive space/speed tradeoff...

Right...so "better" becomes subjective, depends on the use case, so it doesn't make sense to choose -O3 blindly unless you understand the trade-offs and want that side of them for the particular builds you're doing. Things that everyone wants would be in -O2. That's all I'm saying.

Re: Beware of Fast-Math

#23

Post by eqvinox »

nextaccountic wrote: post #20
eqvinox wrote: post #19

Earlier quoted context omitted.

Ah sorry I misunderstood and thought this API was for the other way around, i.e. forbidding "unsafe" operations. (I guess the question reverses to setting those flags) ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D)

> ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D) Okay, the floating point operations are literally algebraic (they form an algebra) but they don't follow some common algebraic properties like associativity. The linked tracking issue itself acknowledges that: > Naming: "algebraic" is not very descriptive of what this does since the operations themsel…

It doesn't feel appropriate to comment there for me not knowing any Rust really, but "lax_" (or "relax_") would have the extra benefit of being very short.

(Is this going to overload operators or are people going to have to type this… a lot… ?)

Re: Beware of Fast-Math

#24

Post by Affric »

For non-associativity what is the best way to order operations? Is there an optimal order for precision whereby more similar values are added/multiplied first?

EDIT: I am now reading Goldberg 1991

Double edit: Kahan Summation formula. Goldberg is always worth going back to.

Re: Beware of Fast-Math

#26

Post by eqvinox »

rlpb wrote: post #22
CamouflagedKiwi wrote: post #15

Earlier quoted context omitted.

The optimisations in -O3 aren't supposed to give incorrect results. They're not in -O2 because they make a more aggressive space/speed tradeoff or increase compile times more significantly. In the same way, the optimisations in -O2 are not meant to be less correct than -O1, but they aren't in that group for similar reasons. -Ofast is the 'dangerous' one. (It includes -ffast-math).

> The optimisations in -O3 aren't supposed to give incorrect results. I didn't mean to imply that they result in incorrect results. > they make a more aggressive space/speed tradeoff... Right...so "better" becomes subjective, depends on the use case, so it doesn't make sense to choose -O3 blindly unless you understand the trade-offs and want that side of them for the particular builds you're doing. Things that everyo…

It doesn't become subjective; things in -O3 can objectively be understood to produce equal or faster code for a higher build cost in the vast majority of cases, roughly averaged across platforms. (Without loss in correctness.)

If you know your exact target and details about your input expectations, of course you can optimize further, which might involve turning off some things in -O3 (or even -O2). On a whole bunch of systems, -Os can be faster than -O3 due to I-cache size limits. But at-large, you can expect -O3 to be faster.

Similar considerations apply for LTO and PGO. LTO is commonly default for release builds these days, it just costs a whole lot of compile time. PGO is done when possible (i.e. known majority inputs).

Re: Beware of Fast-Math

#27

Post by Sharlin »

eqvinox wrote: post #23
nextaccountic wrote: post #20

Earlier quoted context omitted.

> ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D) Okay, the floating point operations are literally algebraic (they form an algebra) but they don't follow some common algebraic properties like associativity. The linked tracking issue itself acknowledges that: > Naming: "algebraic" is not very descriptive of what this does since the operations themsel…

It doesn't feel appropriate to comment there for me not knowing any Rust really, but "lax_" (or "relax_") would have the extra benefit of being very short. (Is this going to overload operators or are people going to have to type this… a lot… ?)

Rust has some precedence for adding convenience newtypes with overloaded operators (eg. `Wrapping´ for `I.wrapping_add(I)` etc). Such a wrapper isn't currently proposed AFAIK but there's no reason one couldn't be added in the future I believe.

Re: Beware of Fast-Math

#28

Post by eqvinox »

Sharlin wrote: post #27
eqvinox wrote: post #23

Earlier quoted context omitted.

It doesn't feel appropriate to comment there for me not knowing any Rust really, but "lax_" (or "relax_") would have the extra benefit of being very short. (Is this going to overload operators or are people going to have to type this… a lot… ?)

Rust has some precedence for adding convenience newtypes with overloaded operators (eg. `Wrapping ´ for `I.wrapping_add(I)` etc). Such a wrapper isn't currently proposed AFAIK but there's no reason one couldn't be added in the future I believe.

Right, as long as the LLVM intrinsics are exposed you could just put that in a crate somewhere.

Re: Beware of Fast-Math

#29

Post by CryZe »

eqvinox wrote: post #23
nextaccountic wrote: post #20

Earlier quoted context omitted.

> ('Naming: "algebraic" is not very descriptive of what this does since the operations themselves are algebraic.' :D) Okay, the floating point operations are literally algebraic (they form an algebra) but they don't follow some common algebraic properties like associativity. The linked tracking issue itself acknowledges that: > Naming: "algebraic" is not very descriptive of what this does since the operations themsel…

It doesn't feel appropriate to comment there for me not knowing any Rust really, but "lax_" (or "relax_") would have the extra benefit of being very short. (Is this going to overload operators or are people going to have to type this… a lot… ?)

WebAssembly also ended up calling its set of similar instructions relaxed.

Re: Beware of Fast-Math

#30

Post by keybored »

Sharlin wrote: post #25

> -funsafe-math-optimizations What's wrong with fun, safe math optimizations?! (:

Hah! I was just about to comment that I immediately read it as fun-safe, everytime I see it.

I guess that happens when I don’t deal with compiler flags daily.

Post reply on HN