Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.
So, what should be used?
Weird compiler bug – Same code, different results
11–20 of 41 posts
Re: Weird compiler bug – Same code, different results
#12Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.
My copy of C11 says Annex F (normative) IEC 60559 floating-point arithmetic F.1 Introduction This annex specifies C language support for the IEC 60559 floating-point standard [...] previously designated ANSI/IEEE 754-1985.
C "supports" whatever the hardware and compiler feel like supporting. It absolutely does not mandate anything. In particular, there are a number of particularly simple compiler optimizations that are not forbidden, though they are not technically correct according to IEEE 754, such as algebraic reassociation and simple commutativity. Moreover, C allows subexpressions to be computed in higher precision (e.g. 80 bit "long double"), which is observable. That last one is primarily due to the x87 FPU coprocessor design that has given us a good 35 years of headaches. Good riddance to that!
Re: Weird compiler bug – Same code, different results
#13I would have never thought that the output of some floating point operations depend on the „reset of the floating point package“. What gives?
At a minimum, when there is a switch between threads the floating point state of the old thread needs to be saved and the floating point state of the incoming thread loaded. This includes such things as rounding mode.
If the thread package was not setting up the "saved" state for a new thread before switching to it then that state could be not what was desired. The best thing to do would probably be to copy the current FP state of the parent at the moment the thread is created.
On a long series of calculations changing the rounding mode could well be enough to cause the difference in results the poster saw.
I would tend to the opinion that redoing the calculation with different rounding modes is in fact a not bad way to figure out how reliable the results are in the first place.
(the rounding mode is not the only thing affected by not saving and restoring the FP state correctly)
Re: Weird compiler bug – Same code, different results
#14Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.
My copy of C11 says Annex F (normative) IEC 60559 floating-point arithmetic F.1 Introduction This annex specifies C language support for the IEC 60559 floating-point standard [...] previously designated ANSI/IEEE 754-1985.
> Support for Annex F (IEEE-754 / IEC 559) of C99/C11
> The Clang compiler does not support IEC 559 math functionality. Clang also does not control and honor the definition of __STDC_IEC_559__ macro. Under specific options such as -Ofast and -ffast-math, the compiler will enable a range of optimizations that provide faster mathematical operations that may not conform to the IEEE-754 specifications. The macro __STDC_IEC_559__ value may be defined but ignored when these faster optimizations are enabled.
If you make use of Clang, you won't find support for Annex F, and in point of fact you can't even macro check to see if it even will support Annex F.
The story with GCC is... More complicated. It guarantees it'll follow Annex F for only some operations [0].
So the upshot is... Most people can't tell when and how Annex F might be followed.
[0] https://gcc.gnu.org/onlinedocs/gcc-7.4.0/gcc/Floating-point-...
Re: Weird compiler bug – Same code, different results
#15[deleted]
Re: Weird compiler bug – Same code, different results
#16Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.
In practice, all of the C compilers will (or can be made to) conform to IEEE 754, although #pragma STDC FENV_ACCESS ON support is very spotty (and consequently non-default rounding-mode support), although Clang/LLVM has been working on this for the past several months.
(That reminds me, I need to harangue the llvm-libc folks to add that pragma to their fpenv code for correctness sake).
Re: Weird compiler bug – Same code, different results
#17Re: Weird compiler bug – Same code, different results
#18Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.
In practice, all of the C compilers will (or can be made to) conform to IEEE 754, although #pragma STDC FENV_ACCESS ON support is very spotty (and consequently non-default rounding-mode support), although Clang/LLVM has been working on this for the past several months. (That reminds me, I need to harangue the llvm-libc folks to add that pragma to their fpenv code for correctness sake).
No, they consistently perform optimizations that violate IEEE 754 rules, such as reassociation and commutation of expressions. These are difficult to observe (unless you are in the habit of observing -0 vs 0), but yes, you can observe them. They are very precisely specified by IEEE 754 and C/C++ compilers perform illegal optimizations.
Re: Weird compiler bug – Same code, different results
#19Earlier quoted context omitted.
In practice, all of the C compilers will (or can be made to) conform to IEEE 754, although #pragma STDC FENV_ACCESS ON support is very spotty (and consequently non-default rounding-mode support), although Clang/LLVM has been working on this for the past several months. (That reminds me, I need to harangue the llvm-libc folks to add that pragma to their fpenv code for correctness sake).
No, they consistently perform optimizations that violate IEEE 754 rules, such as reassociation and commutation of expressions. These are difficult to observe (unless you are in the habit of observing -0 vs 0), but yes, you can observe them. They are very precisely specified by IEEE 754 and C/C++ compilers perform illegal optimizations.
They do those optimizations when you enable fast-math flags. Some compilers (e.g., icc) enable those by default. When fast-math is not enabled, then the IEEE 754-violating optimizations are disabled.
Source: I've been working on such optimizations this past week. And that means establishing precisely which flags I need to have enabled in order for optimization to kick in.
Re: Weird compiler bug – Same code, different results
#20> This means that floating point arithmetic is non-associative. In that A + B != B + A. The equation is the commutative property, not associative. IIRC addition in IEEE-754 is commutative. The property that fails is (A+B)+C = A+(B+C).
Op Here. Yes you are correct. Been very sleep derived when I wrote this (newborns eh). I'll update this.
I didn't want to go too far into the background of floating point math in the post as it wasn't ultimately the issue, just something I was mindful of.