Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU GCC does not round floating-point divisions to the nearest value

lemire.me

Post reply on HN
Print view View original

61–70 of 87 posts

Re: GNU GCC does not round floating-point divisions to the nearest value

#61

Post by tijsvd »

brudgers wrote: post #30

As I understand it, the IEEE floating point standards do not require exact rounding for binary to decimal conversions. See What Every Computer Scientist Should Know About Floating-Point Arithmetic https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h... The behavior appears to be consistent with the standard. Keep in mind that GCC needs to provide the same result irrespective of the architecture it runs on and…

The article is not about binary to decimal conversion. It's about rounding the result of a division, all in binary.

A decimal equivalent would be like saying 200/3 = 66, so rounded towards zero instead of nearest. That is usually not problematic, but it can be in some situations, and in some of those, rounding to nearest can mitigate (for example, when you sum a large number of division results).

The author points out that the behaviour should be configurable, but gcc doesn't seem to honor that.

Re: GNU GCC does not round floating-point divisions to the nearest value

#62

Post by Causality1 »

LambdaComplex wrote: post #56
dx87 wrote: post #49

Earlier quoted context omitted.

For anyone else interested in similar tricks, here is an in-depth video where someone talks about getting a star with half a button press. It starts out straightforward enough, but eventually gets really interesting when they start talking about parallell worlds that you can only access by abusing bugs to reach unrealistic speeds. https://www.youtube.com/watch?v=kpk2tdsPh0A&t=77s

Best followed by this video, which made me laugh my ass off for several minutes: https://www.youtube.com/watch?v=hSa-pwML4z4 (I grew up watching SpongeBob, if that matters)

That was excellent. If I may suggest a followup: https://youtu.be/Pyn3N55elS4

Re: GNU GCC does not round floating-point divisions to the nearest value

#63

Post by jabl »

chrisseaton wrote: post #32
alkonaut wrote: post #29

x87 is crazy. Especially since you can’t easily know when something is truncated from 80 bits to 64. Add a parameter to a function and an unrelated calculation behaves differently because it now ran out of 80bit registers. Congratulations, you now have a spurious test failure you will spend weeks trying to find. The only way of doing FP on x86 and keeping your sanity is by making sure you use 32/64 bit floats 100% of…

I don't think any serious compiler for AMD64 is using x87 - it's ancient history. You would have to go deliberately far out of your way to encounter this problem.

If you use long double on x86-64 Linux, you're using x87. Though that has none of the problems discussed here, as there's no rounding when spilling regs to memory.

Re: GNU GCC does not round floating-point divisions to the nearest value

#64

Post by slavik81 »

saagarjha wrote: post #12

I believe the issue is that some of this ends up evaluated at compile-time, and the compiler really has no idea how to do rounding correctly at that point. When implementing floating point tests for an x86 emulator, we had to create a macro that would ensure that the calculation was done at runtime rather than at compile time for this exact reason: https://github.com/ish-app/ish/blob/18176b6931d69bdabe48f137... . If…

It's been a while since he's done that sort of thing, but Bruce Dawson could probably tell you some stories. I don't remember seeing that specific problem discussed, but his blog is a treasure trove of floating point information: https://randomascii.wordpress.com/category/floating-point/

Re: GNU GCC does not round floating-point divisions to the nearest value

#66

Post by saagarjha »

chongli wrote: post #58
all-fakes wrote: post #41

Earlier quoted context omitted.

I thought fast-math meant (among other things) that the compiler could algebraically simplify things using some rules for real numbers that don't apply to 32-/64-bit floating point numbers. I'd expect that reducing the number of operations would more often than not reduce the error due to rounding, which is what most people would want when they talk about "correctness". Which step in this (very naive) argument is wro…

Rounding error is not the only source of error with floating point. There is also loss of significance, which in the worst case is called catastrophic cancellation [1]. This occurs when subtracting two numbers which are very close in magnitude, for example: 1.23456789 - 1.23456788 = 0.00000001 = 1 * 10^-8 So here we’ve gone from 9 significant figures down to 1. This phenomenon will make a naïve Taylor series approxim…

(Note that in your example no accuracy is lost)

Re: GNU GCC does not round floating-point divisions to the nearest value

#67

Post by josefx »

saagarjha wrote: post #12

I believe the issue is that some of this ends up evaluated at compile-time, and the compiler really has no idea how to do rounding correctly at that point. When implementing floating point tests for an x86 emulator, we had to create a macro that would ensure that the calculation was done at runtime rather than at compile time for this exact reason: https://github.com/ish-app/ish/blob/18176b6931d69bdabe48f137... . If…

> the compiler really has no idea how to do rounding correctly at that point.

Flags, pragmas, function attributes, there are so many ways this could be communicated to the compiler. If this behavior is not a bug, then having a way to change it should be a feature.

Re: GNU GCC does not round floating-point divisions to the nearest value

#68

Post by ndesaulniers »

amluto wrote: post #51
garmaine wrote: post #20

Earlier quoted context omitted.

Why would there be any difference?

Because every x86_64 CPU supports SSE2, so compilers can assume it exists and the ABI passes floating point arguments in SSE2’s FP registers. This means that every sane compiler for x86_64 will use SSE2 and won’t use x87, and the problem won’t occur. In contrast, the 32-bit C ABI passes floating point args in x87 registers, and there are 32-bit CPUs without SSE2, so x87 is used by default. x87 is, in quite a few resp…

For the Linux kernel, things get slightly more complicated with -mgeneral-regs-only. (https://bugs.llvm.org/show_bug.cgi?id=30792 / https://reviews.llvm.org/D38479).

A few drivers use floating point values/calculations. Which way are they rounded when expressions are folded at compile time? Does that match what would happen at runtime?

The x86 kernel also disables SSE generally (kernel_fpu_{begin|end} via -mno-{x87|sse|sse2|....}. This generally lowers the overhead of context switching as there's fewer registers to save+restore, nowadays the SIMD vectors on x86 are huge.

Further, it uses an 8B stack alignment, which makes it so that when FP is used, the compiler cannot select instructions that require 16B aligned operands to be loaded/stored from/to the stack.

Finally, -Ofast implies -funsafe-math. And that's not "fun safe math." Actually had folks at work use "-Ofast" because "why wouldn't I want my code to be fast?" then ask why their reciprocals were also wrong.

Re: GNU GCC does not round floating-point divisions to the nearest value

#69

Post by ndesaulniers »

nwallin wrote: post #16
dpwm wrote: post #5

Earlier quoted context omitted.

My understanding is that in this case it's GCC's optimizer evaluating the expression at compile time in a way that gives a different result to if it's done at runtime. EDIT: GCC gets it wrong with -O0 (when it's evaluated at runtime) and right with -O2 (when it evaluates it at compile time) Clang appears to get it right when evaluated at compile time or runtime. Clang uses the divsd instruction; GCC uses the fdiv ins…

clang gets it "right" because it uses the sse divsd instruction to perform the division, gcc gets it "wrong" because it uses the x87 fdiv instruction. You can configure gcc to use sse math with the -mfpmath=sse instruction. The author states that gcc still gets it "wrong" but he simply is not correct. gcc will use divsd and gets 0.501782303180000055. gcc defaults to use the x87 fpu instead of sse because not all 32 b…

I think `-mno-see -mno-sse2` are of interest here, too.

Re: GNU GCC does not round floating-point divisions to the nearest value

#70

Post by kevingadd »

upofadown wrote: post #37

So the complaint is that this loses half a bit of precision on average? Is that a lot?

If you lose half a bit on every operation and do multiple operations, that could be pretty bad. If it's half a bit at the end of a long chain of operations, that's tolerable and not worse than the precision specified by the docs for some execution environments (GPU shaders, for example)

It also matters a lot whether the error is predictable and possible to compensate for.

Post reply on HN