Demystifying Floating Point Precision
blog.demofox.org
Demystifying Floating Point Precision
1–10 of 57 posts
Re: Demystifying Floating Point Precision
#2I've stated some notes on Floats before, and I think this blogpost is pretty good overall.
The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles):
>>> 1 + 1 + (2. * * 53)
> 9007199254740994.0
>>> 1 + (1 + (2. * * 53))
> 9007199254740992.0
claytonjy noted a point of confusion, the above statement has been edited for clarity.
The number 53 is chosen, because there's 53 bits in the double mantissa. Which is when "+1" will become too small and drop-off of the Double.
In the first case, 1+1 becomes 2 (which is large enough to be added to 2. * * 53). So you end up with the "correct" answer. But (2. * * 53 + 1) rounds down (because the 1 drops off), and the additional +1 afterwards also drops off due to rounding error.
Therefore, Floating Point math is commutative but NOT Associative. And that's really what makes things confusing for most people, in my experience.
x86 with the x87 coprocessor actually performs 80-bit floats to deal with this issue. So you can afford to lose a fair number of bits under older x86 platforms. But modern C++ / C code typically compiles to the faster SSE2 instructions, which only keep 64-bits per operation. This only demonstrates that rounding behavior is not only architecture-specific, but also compiler-option specific !!
Re: Demystifying Floating Point Precision
#3I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
2. 53 should read as 2^53 here (HN markup eaten star characters)
Re: Demystifying Floating Point Precision
#4I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
2. 53 should read as 2^53 here (HN markup eaten star characters)
Thanks, I just edited it.
Re: Demystifying Floating Point Precision
#5I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
good point. That's interesting!
Re: Demystifying Floating Point Precision
#6I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
Is there a way to force 80 bit floats if you need them?
Re: Demystifying Floating Point Precision
#7I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
Is there a way to force 80 bit floats if you need them?
There is a way to politely ask for it and be careful to what you get.
Re: Demystifying Floating Point Precision
#8I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
The second equation confused me a bit as a math-first, cs-second person; looks like a commutative transform, not an associative one. The trick is one must already trust these ops are commutative.
The second equation might be more clearly written as: >>> 1 + (1 + (2. * * 53)) to avoid hiding the associativity behind the assumption of commutativity.
Re: Demystifying Floating Point Precision
#9I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
Is there a way to force 80 bit floats if you need them?
If you want a guarantee, use something like gmp that'll do them in software.
Re: Demystifying Floating Point Precision
#10I've stated some notes on Floats before, and I think this blogpost is pretty good overall. The one thing that trips up a lot of people (that wasn't mentioned in this post) is that Floats are non-associative. Try the following in Python (or whatever language that uses Doubles): >>> 1 + 1 + (2. * * 53) > 9007199254740994.0 >>> 1 + (1 + (2. * * 53)) > 9007199254740992.0 claytonjy noted a point of confusion, the above st…
I believe the x87 uses 80 bits internally, but it only runs operations until it has sufficient precision to do the rounding (which is hard to do in general, see: http://perso.ens-lyon.fr/jean-michel.muller/Intro-to-TMD.htm)