There was another good article posted on HN on floating points - http://lemire.me/blog/2017/02/28/how-many-floating-point-num...
where the key takeaway is that between -1 -> 1 is where the %50 of floats live.
Comparing Floating-Point Numbers Is Tricky
31–40 of 45 posts
Re: Comparing Floating-Point Numbers Is Tricky
#32The operator overlooks my pet peeve when it comes to comparing floating-point numbers: in C++ the "standard associative containers" (std::set and std::map) are based on ordering using an less-than relationship which must be a "strict weak ordering". Many times, the methods suggested for comparing floating point types do not satisfy the requirements of "strict weak ordering", and in this case the C++ standard says you…
Ah! fond memories, related one of the most vexing and entertaining bug that I discovered in my code.
I was using C++'s std::sort and soon enough things would go wrong. It would crash at unpredictable times, I suspected I was corrupting memory somewhere. Parts of my data structures would get overwritten by parts from some other data structure. I checked and checked and checked my code, nothing seemed wrong.
Its only after opening the covers, when I started peering into the sort that I realized my mistake. I was passing the comparison operator that was a "less than equal" relation.
Re: Comparing Floating-Point Numbers Is Tricky
#33Earlier quoted context omitted.
Floating-point math is carefully-defined. There are multiple independently-developed but interoperable implementations and an IEEE standard that talks in detail about how floating-point math is supposed to work. It's not "a bit fuzzy."
It most certainly is a bit fuzzy. For a fun critique of the newer IEEE standard by a real FP expert, see: http://www.russinoff.com/papers/ieee.pdf For something more concrete, consider Section 8, "Variations Allowed by the IEEE Floating-Point Standard", of the TestFloat tool for testing floating point implementations for IEEE compliance: http://www.jhauser.us/arithmetic/TestFloat-3c/doc/TestFloat-... And of course, m…
I wasn't aware of the ambiguity surrounding the underflow flag noted in your second link. However, the other complaints I'm reading from your links (things not specified by the standard may vary in behaviour; the standard is written in English and could have been written in different or better English) do not impact the semantics of floating-point arithmetic in a material way.
Re: Comparing Floating-Point Numbers Is Tricky
#34There was a very good article explaining this using MATLAB, but I can't find it right now. This one is pretty close and explains the concepts of overflow, underflow, etc. The diagrams about "eps" are pretty good, even if your language of choice is Python, C/C++, etc. http://blogs.mathworks.com/cleve/2014/07/07/floating-point-n...
Python has math.isclose() in the standard library, with configurable tolerances:
Re: Comparing Floating-Point Numbers Is Tricky
#35The operator overlooks my pet peeve when it comes to comparing floating-point numbers: in C++ the "standard associative containers" (std::set and std::map) are based on ordering using an less-than relationship which must be a "strict weak ordering". Many times, the methods suggested for comparing floating point types do not satisfy the requirements of "strict weak ordering", and in this case the C++ standard says you…
This is actually caused by the use of the x87 80 bit floating point registers. (Infamous GCC bug #323) When the float is first inserted into the set/map it has 80 bit precision, but that gets truncated to float or double precision during the store. This breaks the ordering as you are saying, but it's not an inherent flaw with floats as such. The problem goes away if you compile with -mfpmath=sse because then the math…
As bug #323 points out, truncating floats is an incomplete solution and brings its own problems. The GNU people were not simply being lazy or ignorant, their approach was valid.
The "bug" is x87 design, period. And x87 is the past. It's old, and bad. SSE and IEEE 754 is the present.
Re: Comparing Floating-Point Numbers Is Tricky
#36Comparing floating point numbers for equality us tricky. In fact it is a classic fool's errand. Comparing for lesser or greater is not tricky.
Comparing for equality is not tricky because it works:
3.1415927410125732421875 equals 3.1415927410125732421875
The problem is that programmers expect the float to be a decimal (smaller than the float) or integer.
For example if you want to calculate that two vectors are parallel then comparing for lesser or greater won't give you the expected result if you need them to be zero in difference. You might think that the vectors (0, 10) and (10, 10) are parallel but using !(x > 0 || x < 0) won't help you. You still end up using a threshold of the smallest float, something like (abs(x - 0) < 0.0000000000000000000001)
Re: Comparing Floating-Point Numbers Is Tricky
#37I try to always remember:
Floats are great for quick, mostly correct, math.
Think REALLY carefully about any process that then 'compares' the result. Usually you're "doing it wrong" if that's the case.
I think I might even find it useful if compilers could be instructed to warn whenever comparison operators were used on a float type value.
Re: Comparing Floating-Point Numbers Is Tricky
#38My basic rule of thumb is to only use floats at the presentation layer, for storage, or for measurements. If you're doing any serious calculations you need to normalise to a more reliable format first.
Re: Comparing Floating-Point Numbers Is Tricky
#39Earlier quoted context omitted.
Floating-point math is carefully-defined. There are multiple independently-developed but interoperable implementations and an IEEE standard that talks in detail about how floating-point math is supposed to work. It's not "a bit fuzzy."
It most certainly is a bit fuzzy. For a fun critique of the newer IEEE standard by a real FP expert, see: http://www.russinoff.com/papers/ieee.pdf For something more concrete, consider Section 8, "Variations Allowed by the IEEE Floating-Point Standard", of the TestFloat tool for testing floating point implementations for IEEE compliance: http://www.jhauser.us/arithmetic/TestFloat-3c/doc/TestFloat-... And of course, m…
The problem with the "floating point is fuzzy" comment is that people start treating it as some sort of black box, or as if the results are random somehow. Sure there are a few weird things with floating point status flags, but mostly "fuzziness" is perfectly understandable when you grasp what it is doing (including the usual 0.1 + 0.2 != 0.3).
Also the standard does specify trig functions (§9.2 Recommended correctly rounded functions), but that's one of the optional parts, and as far as I know no one has actually implemented them fully (CRlibm came close, but I don't think their pow function has been fully proven to be correctly rounded, and in any case it isn't widely used).
This is actually a big problem with most standards: a lot of them contain finicky details about which only a very small subset of people care. As far as I know, there still isn't a C compiler that implements all the PRAGMAs specified in the C-1999/C-2011 specs.
Re: Comparing Floating-Point Numbers Is Tricky
#40Here's my less-than-scientific floating point near-equality test I use.
bool zero(float x) ( return x*x Critique and comments welcome.