31–40 of 117 posts
The question is how to enable all this in Rust... Right now there's no simple way to just tell Rust to be fast & loose with floats.
If you use the LLVM D compiler you can opt in or out of these individually on a per-function basis. I don't trust them globally.
with gcc you can also use #pragma gcc optimize or __attribute__(optimize(...)) for a similar effect. It is not 100% bug free (at least it didn't use to) and often it prevents inlining a function into another having different optimization levels (so in practice its use has to be coarse grained).
https://gcc.godbolt.org/z/voMK7x7hG
Try it with and without the pragma, and adding -ffast-math to the compiler command line. It seems that with the pragma sqrt(x) * sqrt(x) becomes sqrt(x*x), but with the command line version it is simplified to just x.
Earlier quoted context omitted.
> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.
And don't do any division. Easy.
I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…
Generally if you're seeing a NaN/Inf something has gone wrong, It's very difficult to gracefully recover from and if you tried I think you would lose both sanity and performance! Regarding performance, the cost of a real division is about 3-4 orders worse performance than an if statement that is very consistent, but the usual way is to have fast/safe versions of functions, where you need performance and can deduce if…
That’s a bold claim.
I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…
Do you mean `if-else if` or `if-else`? Because the second would work and would always short-circuit to else.
Earlier quoted context omitted.
Denormal floats are not a purpose, they are not used intentionally. When the CPU generates denormal floats on underflow, that ensures that underflow does not matter, because the errors remain the same as at any other floating-point operation. Without denormal floats, underflow must be an exception condition that must be handled somehow by the program, because otherwise the computation errors can be much higher than e…
Thanks for this explanation. But yeah, I meant : are there applications where use of denormals vs. flush-to-zero is actually useful... ? If your variable will be nearing zero, and e.g. you use it as a divider, don't you need to handle a special case anyway ? Just like you should handle your integer overflows. I'm seeing denormals as an extension of the floating point range, but with a tradeoff that's not worth it. Ma…
Without denormals a check before a division may fail to detect a division by 0.
Earlier quoted context omitted.
He means simple stuff like ensuring a function like float unit_price (float volume, int count) { return volume / count; } is only called where count > 0
But couldn’t the compiler optimize out the check because you’re guaranteeing to never divide by 0, so logically count would never be 0. Akin to undefined behavior almost.
if (count > 0)
total += unit_price(volume, count);
total += unit_price(volume, count);
if (count == 0)
printf("oops\n");
Earlier quoted context omitted.
> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.
And don't do any division. Easy.
I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used. I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So opti…
> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.
The workflow would then be:
1. prove that the input is not null (if input == 0) yielding a "not null number"
2. running your calculation (no check needed for multiplication or division, if you add something you have to reprove that the number is not null)Earlier quoted context omitted.
Denormal floats are not a purpose, they are not used intentionally. When the CPU generates denormal floats on underflow, that ensures that underflow does not matter, because the errors remain the same as at any other floating-point operation. Without denormal floats, underflow must be an exception condition that must be handled somehow by the program, because otherwise the computation errors can be much higher than e…
Thanks for this explanation. But yeah, I meant : are there applications where use of denormals vs. flush-to-zero is actually useful... ? If your variable will be nearing zero, and e.g. you use it as a divider, don't you need to handle a special case anyway ? Just like you should handle your integer overflows. I'm seeing denormals as an extension of the floating point range, but with a tradeoff that's not worth it. Ma…
You must keep in mind that if you enable flush-to-zero and you really see an increase in speed, that means that you have now injected errors in your FP computations. Whether the errors matter or not, that depends on the application.
If you do not see any serious speed improvement, then you have no reason to enable flush-to-zero. Even when you might want to use flush-to-zero in a production binary, you should not use it during development.
Prior to enabling flush-to-zero it would be good to enable trap-on-underflow instead of denormals, to see if this really happens frequently enough to matter for performance and possibly to investigate the reason, to see if underflow could be avoided in other ways.
In conclusion using denormals is always useful, because it limits the errors accumulated during FP computation. On some CPUs you may get higher speed by not using denormals, but only if large errors do not matter. If you avoid underflows by other means, e.g. range checking of inputs, then it does not matter how slow the operations with denormals are, so there is no reason to use any option that enables flush-to-zero (e.g. -ffast-math).
Making a FP computation with "double" numbers and enabling flush-to-zero on that would usually be quite stupid, because if the errors do not matter, then that is a task for single-precision "float" (unless you need "double" only for the exponent range, not for precision).