Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU GCC does not round floating-point divisions to the nearest value

lemire.me

Post reply on HN
Print view View original

51–60 of 87 posts

Re: GNU GCC does not round floating-point divisions to the nearest value

#51

Post by amluto »

garmaine wrote: post #20
amluto wrote: post #13

Perhaps the title should mention that this is 32-bit x86, which is rather obsolete. This should work okay on x86_64.

Why would there be any difference?

Because every x86_64 CPU supports SSE2, so compilers can assume it exists and the ABI passes floating point arguments in SSE2’s FP registers. This means that every sane compiler for x86_64 will use SSE2 and won’t use x87, and the problem won’t occur.

In contrast, the 32-bit C ABI passes floating point args in x87 registers, and there are 32-bit CPUs without SSE2, so x87 is used by default.

x87 is, in quite a few respects, awful.

Re: GNU GCC does not round floating-point divisions to the nearest value

#52

Post by kevin_thibedeau »

bonzini wrote: post #44
all-fakes wrote: post #41

Earlier quoted context omitted.

I thought fast-math meant (among other things) that the compiler could algebraically simplify things using some rules for real numbers that don't apply to 32-/64-bit floating point numbers. I'd expect that reducing the number of operations would more often than not reduce the error due to rounding, which is what most people would want when they talk about "correctness". Which step in this (very naive) argument is wro…

Stuff like Kahan summation breaks horribly with -ffast-math, because if you assume that sum is associative the error term, which is ((s+x)-s)-x, simplifies to zero. https://en.m.wikipedia.org/wiki/Kahan_summation_algorithm

That's where you dust off volatile.

Re: GNU GCC does not round floating-point divisions to the nearest value

#53

Post by sonofgod »

jmblpati wrote: post #43
sonofgod wrote: post #15

An example where this matters -- a new Mario 64 trick on the Virtual Console version because summing round-towards-zero numbers over a sine wave cycle produces a small delta which builds up over a couple of days to make a jump possible https://www.kotaku.com.au/2018/06/the-mario-64-trick-that-ta...

A small clarification: this trick is not new. It was discovered very early on in the investigation of the Wii VC release of Super Mario 64. It was rediscovered by a group of people who were interested in an application of it (which enables one to beat SM64 without the A button) a couple of years ago, and the discovery that the glitch is a result of inaccurate rounding mode emulation followed a couple weeks later.

Thank you for the clarification.

Re: GNU GCC does not round floating-point divisions to the nearest value

#54

Post by sonofgod »

dx87 wrote: post #49
sonofgod wrote: post #15

An example where this matters -- a new Mario 64 trick on the Virtual Console version because summing round-towards-zero numbers over a sine wave cycle produces a small delta which builds up over a couple of days to make a jump possible https://www.kotaku.com.au/2018/06/the-mario-64-trick-that-ta...

For anyone else interested in similar tricks, here is an in-depth video where someone talks about getting a star with half a button press. It starts out straightforward enough, but eventually gets really interesting when they start talking about parallell worlds that you can only access by abusing bugs to reach unrealistic speeds. https://www.youtube.com/watch?v=kpk2tdsPh0A&t=77s

Strongly recommended watching, in my opinion: very clearly describes some quite complicated manipulations, the circumstances in which they work and how to move between those circumstances, to achieve what should be impossible.

If you're going to watch one commentated "tool-assisted" run, this is it.

Re: GNU GCC does not round floating-point divisions to the nearest value

#55

Post by olliej »

bonzini wrote: post #46
olliej wrote: post #28

Earlier quoted context omitted.

The platform ABI can differ on the same cpu. I originally had the wrong answer here, but I'm including below for historical purposes and embarrassment. Re-reading on my laptop meant I didn't misread/read-the-expected? Basically by default when compiling for IA32 gcc assumes that there is no SSE unit, and so must use the x87 fpu. Alas the x87 unit doesn't have either 32 or 64 bit ieee floating point types, so the calc…

> Anyway it is not sound to round twice in any numeric operation, and by doing the calculation at one precision and then rounding that to the required precision that is what gcc is doing. > > This is a bug in GCC, floating point is well defined, there isn’t randomness to it, and they’ve introduced additional rounding that is not valid. Except the C standard has a way to communicate how the extra rounding is being int…

as my edit said the problem was gcc was confined to using x87 as the compile args required x87 - the older version of the comment incorrectly thought it was the optimizer going wrong, rather than runtime.

If you are constrained to x87 your only option is multiple roundings which is bad. Ideally you'd just keep processing in the x87 stack as long as possible, but that would result in a different version of "incorrect" results.

Honestly if you're a compiler required to use x87 with non-80bit floating point I'm not sure how you can do the right thing :-/

I don't know what gcc has but llvm has a full software floating point implementation to handle cross compiling so it can match native arithmetic.

Re: GNU GCC does not round floating-point divisions to the nearest value

#56

Post by LambdaComplex »

dx87 wrote: post #49
sonofgod wrote: post #15

An example where this matters -- a new Mario 64 trick on the Virtual Console version because summing round-towards-zero numbers over a sine wave cycle produces a small delta which builds up over a couple of days to make a jump possible https://www.kotaku.com.au/2018/06/the-mario-64-trick-that-ta...

For anyone else interested in similar tricks, here is an in-depth video where someone talks about getting a star with half a button press. It starts out straightforward enough, but eventually gets really interesting when they start talking about parallell worlds that you can only access by abusing bugs to reach unrealistic speeds. https://www.youtube.com/watch?v=kpk2tdsPh0A&t=77s

Best followed by this video, which made me laugh my ass off for several minutes: https://www.youtube.com/watch?v=hSa-pwML4z4

(I grew up watching SpongeBob, if that matters)

Re: GNU GCC does not round floating-point divisions to the nearest value

#57

Post by bonzini »

olliej wrote: post #55
bonzini wrote: post #46

Earlier quoted context omitted.

> Anyway it is not sound to round twice in any numeric operation, and by doing the calculation at one precision and then rounding that to the required precision that is what gcc is doing. > > This is a bug in GCC, floating point is well defined, there isn’t randomness to it, and they’ve introduced additional rounding that is not valid. Except the C standard has a way to communicate how the extra rounding is being int…

as my edit said the problem was gcc was confined to using x87 as the compile args required x87 - the older version of the comment incorrectly thought it was the optimizer going wrong, rather than runtime. If you are constrained to x87 your only option is multiple roundings which is bad. Ideally you'd just keep processing in the x87 stack as long as possible, but that would result in a different version of "incorrect"…

Yes, FLT_EVAL_METHOD 2 is an extension of "keep values on the stack as much as possible" that also requires the compiler to spill intermediate results as long doubles. Apparently it also doesn't cover constant folding, but knowing who implemented it in GCC (Joseph Myers) he's extremely thorough and I have no doubt it's allowed by the standard.

That's the best they can do.

Re: GNU GCC does not round floating-point divisions to the nearest value

#58

Post by chongli »

all-fakes wrote: post #41
garmaine wrote: post #23

Earlier quoted context omitted.

It’s normally not more correct.

I thought fast-math meant (among other things) that the compiler could algebraically simplify things using some rules for real numbers that don't apply to 32-/64-bit floating point numbers. I'd expect that reducing the number of operations would more often than not reduce the error due to rounding, which is what most people would want when they talk about "correctness". Which step in this (very naive) argument is wro…

Rounding error is not the only source of error with floating point. There is also loss of significance, which in the worst case is called catastrophic cancellation [1]. This occurs when subtracting two numbers which are very close in magnitude, for example:

1.23456789 - 1.23456788 = 0.00000001 = 1 * 10^-8

So here we’ve gone from 9 significant figures down to 1. This phenomenon will make a naïve Taylor series approximation of e^x be very inaccurate for negative x, due to the sign alternating between positive and negative on every term, causing a lot of catastrophic cancellation.

[1] https://en.wikipedia.org/wiki/Loss_of_significance

Re: GNU GCC does not round floating-point divisions to the nearest value

#59

Post by olliej »

bonzini wrote: post #57
olliej wrote: post #55

Earlier quoted context omitted.

as my edit said the problem was gcc was confined to using x87 as the compile args required x87 - the older version of the comment incorrectly thought it was the optimizer going wrong, rather than runtime. If you are constrained to x87 your only option is multiple roundings which is bad. Ideally you'd just keep processing in the x87 stack as long as possible, but that would result in a different version of "incorrect"…

Yes, FLT_EVAL_METHOD 2 is an extension of "keep values on the stack as much as possible" that also requires the compiler to spill intermediate results as long doubles. Apparently it also doesn't cover constant folding, but knowing who implemented it in GCC (Joseph Myers) he's extremely thorough and I have no doubt it's allowed by the standard. That's the best they can do.

yeah, once you are restricted to performing (at runtime) computation at a precision that does not match the specified storage precision, the compiler is kind of screwed :-/
Post reply on HN