101–110 of 152 posts
It feels backwards that you need to order your comparisons so as to generate optimal assembly.
Earlier quoted context omitted.
Why do you say almost never? Don’t let the name scare you; all floating point math is inaccurate. Fast math is only slightly less accurate, I think typically it’s a 1 or maybe 2 LSB difference. At least in CUDA it is, and I think many (most?) people & situations can tolerate 22 bits of mantissa compared to 23, and many (most?) people/situations aren’t paying attention to inf/nan/exception issues at all. I deal with a…
Here are some of the problems with fast-math: * It links in an object file that enables denormal flushing globally, so that it affects all libraries linked into your application, even if said library explicitly doesn't want fast-math. This is seriously one of the most user-hostile things a compiler can do. * The results of your program will vary depending on the exact make of your compiler and other random attributes…
This already shouldn't be assumed, because even the same code, compiler, and flags can produce different floating point results on different CPU targets. With the world increasingly split over x86_64 and aarch64, with more to come, it would be unwise to assume they produce the same exact numbers.
Often this comes down to acceptable implementation defined behavior, e.g. temporarily using an 80-bit floating register despite the result being coerced to 64 bits, or using an FMA instruction that loses less precision than separate multiply and add instructions.
Portable results should come from integers (even if used to simulate rationals and fixed point), not floats. I understand that's not easy with multiplayer games, but doing so with floats is simply impossible because of what is left as implementation-defined in our language standards.
Earlier quoted context omitted.
That was what Reagan said about the Soviet Union, not what was said in the Soviet Union. Correct me if I'm wrong.
https://en.wikipedia.org/wiki/Trust,_but_verify > Trust, but verify (Russian: доверяй, но проверяй, tr. doveryay, no proveryay, IPA: [dəvʲɪˈrʲæj no prəvʲɪˈrʲæj]) is a Russian proverb, which is rhyming in Russian. The phrase became internationally known in English after Suzanne Massie, a scholar of Russian history, taught it to Ronald Reagan, then president of the United States, the latter of whom used it on several o…
Earlier quoted context omitted.
That’s fair. Ideally transitive dependencies should be completely hidden from you. Hopefully the author of the library you include directly has heeded the instructions of libraries they depend on. Hey I grant and acknowledge that using fast-math carries a little risk of surprises, we don’t necessarily need to try to think of corner cases. I’m mostly pushing back a little because using floats at all carries almost as…
> A lot of people seem to use floats without knowing how inaccurate floats are, Small nit, but floats aren't inaccurate, they have non uniform precision. Some float operations can be inaccurate, but that's rather path dependent... One problem with -ffast-math is that a) it sounds appealing and b) people don't understand floats, so lots of people turn it on without understanding what it does, and that can introduce su…
Earlier quoted context omitted.
Thank you, great points. I’d have to agree that disabling denorms globally is pretty bad, even if (or maybe especially if) caring about denorms is rare. > Fast-math can cause hard range guarantees to fail. Maybe you’ve got code that you can prove that, even with rounding error, the result will still be >= 0. Floats do this too, it’s pretty routine to bump into epsilon out-of-range issues without fast-math. Most peopl…
> I’d have to agree that disabling denorms globally is pretty bad, and yet, for audio processing, this is an option that most DAWs either implement silently, or offer users the choice, because denormals are inevitable in reverb tails and on most Intel processors they slow things by orders of magnitude.
Depending on the order of the arguments to min max you'll get an extra move instruction [1]: std::min(max, std::max(min, v)); maxsd xmm0, xmm1 minsd xmm0, xmm2 std::min(std::max(v, min), max); maxsd xmm1, xmm0 minsd xmm2, xmm1 movapd xmm0, xmm2 For min/max on x86 if any operand is NaN the instruction copies the second operand into the first. So the compiler can't reorder the second case to look like the first (to lea…
Earlier quoted context omitted.
Here are some of the problems with fast-math: * It links in an object file that enables denormal flushing globally, so that it affects all libraries linked into your application, even if said library explicitly doesn't want fast-math. This is seriously one of the most user-hostile things a compiler can do. * The results of your program will vary depending on the exact make of your compiler and other random attributes…
> The results of your program will vary depending on the exact make of your compiler and other random attributes of your compile environment, which can wreak havoc if you have code that absolutely wants bit-identical results. This doesn't matter for everybody, but there are some domains where this can be a non-starter (e.g., multiplayer game code). This already shouldn't be assumed, because even the same code, compil…
I believe this is "always" rather than often when it comes to the actual operations defined by the FP standard. gcc does play it fast and loose (as -ffast-math is not yet enabled by default, and FMA on the other hand is), but this is technically illegal and at least can be easily configured to be in standards-compliant mode.
I think the bigger problem comes from what is _not_ documented by the standard. E.g. transcendental functions. A program calling plain old sqrt(x) can find itself behaving differently _even between different stepping of the same core_, not to mention that there are well-known differences between AMD vs Intel. This is all using the same binary.
Earlier quoted context omitted.
Here are some of the problems with fast-math: * It links in an object file that enables denormal flushing globally, so that it affects all libraries linked into your application, even if said library explicitly doesn't want fast-math. This is seriously one of the most user-hostile things a compiler can do. * The results of your program will vary depending on the exact make of your compiler and other random attributes…
> The results of your program will vary depending on the exact make of your compiler and other random attributes of your compile environment, which can wreak havoc if you have code that absolutely wants bit-identical results. This doesn't matter for everybody, but there are some domains where this can be a non-starter (e.g., multiplayer game code). This already shouldn't be assumed, because even the same code, compil…
All CPU hardware nowadays conforms to IEEE 754 semantics for binary32 and binary64. (I think all the GPUs now have non-denormal-flushing modes, but my GPU knowledge is less deep). All compilers will have a floating-point mode that preserves IEEE 754 semantics assuming that FP exceptions are unobservable and rounding mode is the default, and this is usually the default (icc/icx is unusual in making fast-math the default).
Thus, you have portability of floating-point semantics, subject to caveats:
* The math library functions [1] are not the same between different implementations. If you want portability, you need to ensure that you're using the exact same math library on all platforms.
* NaN payloads are not consistent on different platforms, or necessarily within the same platform due to compiler optimizations. Note that not even IEEE 754 attempts to guarantee NaN payload stability.
* Long double is not the same type on different platforms. Don't use it. Seriously, don't.
* 32-bit x86 support for exact IEEE 754 equivalence is essentially a "known-WONTFIX" bug. (This is why the C standard implemented FLT_EVAL_METHOD). The x87 FPU evaluates everything in 80-bit precision, and while you can make this work for binary32 easily (double rounding isn't an issue), though with some performance cost (the solution involves reading/writing from memory after every operation), it's not so easy for binary64. However, the SSE registers do implement IEEE 754 exactly, and are present on every chip old enough to drink, so it's not really a problem anymore. There's a subsidiary issue that the x86-32 ABI requires floats be returned in x87 registers, which means you can't properly return an sNaN correctly, but sNaN and floating-point exceptions are firmly in the realm of nonportability anyways.
In short, if you don't need to care about 32-bit x86 support (or if you do care but can require SSE2 support), and you don't care about NaNs, and you bring your own libraries along, you can absolutely expect to have floating-point portability.
[1] It's actually not even all math library functions, just those that are like sin, pow, exp, etc., but specifically excluding things like sqrt. I'm still trying to come up with a good term to encompass these.
Earlier quoted context omitted.
> The results of your program will vary depending on the exact make of your compiler and other random attributes of your compile environment, which can wreak havoc if you have code that absolutely wants bit-identical results. This doesn't matter for everybody, but there are some domains where this can be a non-starter (e.g., multiplayer game code). This already shouldn't be assumed, because even the same code, compil…
This advice is out-of-date. All CPU hardware nowadays conforms to IEEE 754 semantics for binary32 and binary64. (I think all the GPUs now have non-denormal-flushing modes, but my GPU knowledge is less deep). All compilers will have a floating-point mode that preserves IEEE 754 semantics assuming that FP exceptions are unobservable and rounding mode is the default, and this is usually the default (icc/icx is unusual i…
Transcendental functions. They're called that because computing an exactly rounded result might be unfeasible for some inputs. https://en.wikipedia.org/wiki/Table-maker%27s_dilemma So standards for numerical compute punt on the issue and allow for some error in the last digit.
Earlier quoted context omitted.
This advice is out-of-date. All CPU hardware nowadays conforms to IEEE 754 semantics for binary32 and binary64. (I think all the GPUs now have non-denormal-flushing modes, but my GPU knowledge is less deep). All compilers will have a floating-point mode that preserves IEEE 754 semantics assuming that FP exceptions are unobservable and rounding mode is the default, and this is usually the default (icc/icx is unusual i…
> It's actually not even all math library functions, just those that are like sin, pow, exp, etc., but specifically excluding things like sqrt. I'm still trying to come up with a good term to encompass these. Transcendental functions. They're called that because computing an exactly rounded result might be unfeasible for some inputs. https://en.wikipedia.org/wiki/Table-maker%27s_dilemma So standards for numerical com…
(The main defining factor is if they're an IEEE 754 §5 operation or not, but IEEE 754 isn't a freely-available standard.)