Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Examples of floating point problems

jvns.ca

Post reply on HN
Print view View original

131–140 of 179 posts

Re: Examples of floating point problems

#131

Post by bachmeier »

WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This is why D (in one of its more controversial choices) sets uninitialized floating point values to NaN rather than the more conventional choice of 0.0.

This is definitely something I like about D, but I'd much prefer a compiler error. double x; double y = x+1.5; is less than optimal.

Re: Examples of floating point problems

#132

Post by defrost »

svat wrote: post #120
defrost wrote: post #98

Earlier quoted context omitted.

> whether we can rely on (say) the operation 2.0 + 3.0 giving exactly 5.0 (we can!) Can we rely upon 2.3 + 2.3 giving exactly 4.6 though? Can we rely upon LargeInt.0 + LargeInt.0 giving exactly 2xLargeInt.0 for all integers?

That's exactly my point: when you internalize the diagram, you'll be able to reason confidently about what happens: • In the case of "2.3 + 2.3", each "2.3" is “snapped” to the nearest representable value (green line in the diagram), then their sum snapped to the nearest green line. In this case, because the two summands are equal and we're using binary floating-point, the result will also be a green line. If you kne…

> Again, with the mental model, you'll be in a better position to state what you expect by "exactly".

I'm long in the tooth, if I want exactly I'll use a symbolic algebra program (such as Cayley|Magma).

I don't expect exactly when using floats (or doubles, etc) and my mental model includes the image of the "fine teeth of coverage" having uneven spacing, that the float graticule across the reals is uneven and at best semi quasi regular has largely been my greatest issue (in geophysical computation engine development).

Re: Examples of floating point problems

#133

Post by WalterBright »

gerdesj wrote: post #116
jordigh wrote: post #72

Earlier quoted context omitted.

> What's the result of 1.0/0.0? It's not a number, so it's a NaN It's not often that I get to correct Mr D himself, but 1.0/0.0 is...

I often find myself wondering wtf? when I see discussions like this. root -1 is i - and we get into complex numbers. If I ever end up with needing to deal with a square root of a number that might be -1 then I'll do it the old fashioned way and test for that and sort it. What is the problem here? root -1 is well defined! Equally (lol) 1.0/0.0 and 1/0 can be tricky to handle but not beyond whit of person. In those cas…

> root -1 is well defined!

Yes, if you're using complex number types. Nope if you're using reals:

    double x = sqrt(1.0);

Re: Examples of floating point problems

#134

Post by gautamcgoel »

maximilianburke wrote: post #112
BeetleB wrote: post #109

Earlier quoted context omitted.

> Both C and C++. Seems C++ only added it in C++11. Surprising. > That's why I said in the second part that we need to do away with them. Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down. I'd wager that for the majority of programs written, the fast math is as goo…

> I'd wager that for the majority of programs written, the fast math is as good as the accurate math. I'd take that wager. I spent 13 years working on video games and video game technology, a domain where floating point performance is critical, and by and large we never used fast-math because of the problems it created for us.

This is surprising to me! Can you explain what problems you encountered? My (limited) understanding is that the main effect of fast-math is to turn off support for subnormals. Since subnormals are only used to represent extremely small values, I wouldn't expect them to have much effect in the real world.

Re: Examples of floating point problems

#135

Post by a_e_k »

UncleMeat wrote: post #119
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> The result of any operation on a NaN is a NaN. That's not true! maxNum(nan, x) = x.

The standard for comparison with NaNs is to always return false [0].

So that entirely depends on how max() is implemented. A naive implementation of max() might just as easily instead return NaN for that. Or if max(NaN, x) is x, then it may give NaN for max(x, NaN).

Note that the fact that comparisons always return false also means that sorting an array of floating point values that contain NaNs can very easily break a comparison-based sorting algorithm!! (I've seen std::sort() in C++, for example, crash because NaNs break the strict weak ordering [1] requirement.)

[0] https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN

[1] https://en.cppreference.com/w/cpp/named_req/Compare

Re: Examples of floating point problems

#136

Post by stephencanon »

BeetleB wrote: post #129
subharmonicon wrote: post #115

Earlier quoted context omitted.

isnan is a macro in C, function in C++. But either way, it’s an abstraction that provides a standard test for NaNs in those languages.

I believe the reason he is pointing out this distinction is that if it is a macro, it can be optimized out causing bugs.

Not in a sound implementation. E.g. in macOS/iOS’s libm, we replace the isnan() macro with a function call if you compile with fast-math precisely to prevent this.

Re: Examples of floating point problems

#137

Post by a_e_k »

avgcorrection wrote: post #125
svat wrote: post #120

Earlier quoted context omitted.

That's exactly my point: when you internalize the diagram, you'll be able to reason confidently about what happens: • In the case of "2.3 + 2.3", each "2.3" is “snapped” to the nearest representable value (green line in the diagram), then their sum snapped to the nearest green line. In this case, because the two summands are equal and we're using binary floating-point, the result will also be a green line. If you kne…

> If you knew more about the details of binary64 aka float64, you could confidently say that "2.3" means 2.29999995231628417969 […]" Then I think writing let f = 2.3; Should be a compile error. The compiler should force you to write the “snapped” value in order to not mislead. :)

Now try writing the "snapped" value for 2.3 as a finite decimal. :-)

Re: Examples of floating point problems

#138

Post by subharmonicon »

BeetleB wrote: post #129
subharmonicon wrote: post #115

Earlier quoted context omitted.

isnan is a macro in C, function in C++. But either way, it’s an abstraction that provides a standard test for NaNs in those languages.

I believe the reason he is pointing out this distinction is that if it is a macro, it can be optimized out causing bugs.

Sure with -fast-math or similar compiler options without a way to also say “preserve NaNs”, that could happen with macros, but it can also happen with inline functions where the body is in a header file.

Re: Examples of floating point problems

#139

Post by subharmonicon »

marcosdumay wrote: post #114
BeetleB wrote: post #109

Earlier quoted context omitted.

> Both C and C++. Seems C++ only added it in C++11. Surprising. > That's why I said in the second part that we need to do away with them. Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down. I'd wager that for the majority of programs written, the fast math is as goo…

> Some applications are too slow without them. I fully expect the intersection between programs where floating point instructions are a speed bottleneck and programs where the numerical instability that fast_math can cause is not a problem is the empty set.

Something akin to fast-math is the default for some shading languages (although some have switched to fast-math-but-preserve-NaN-and-INF).

Re: Examples of floating point problems

#140

Post by subharmonicon »

gautamcgoel wrote: post #134
maximilianburke wrote: post #112

Earlier quoted context omitted.

> I'd wager that for the majority of programs written, the fast math is as good as the accurate math. I'd take that wager. I spent 13 years working on video games and video game technology, a domain where floating point performance is critical, and by and large we never used fast-math because of the problems it created for us.

This is surprising to me! Can you explain what problems you encountered? My (limited) understanding is that the main effect of fast-math is to turn off support for subnormals. Since subnormals are only used to represent extremely small values, I wouldn't expect them to have much effect in the real world.

fast-math can result in many things you may not expect, e.g. treating FP operations as being associative and distributive, which they aren’t.
Post reply on HN