Live data from Hacker News

Examples of floating point problems

jvns.ca

121–130 of 179 posts

Re: Examples of floating point problems

#121

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> But let's study it a bit. Suppose you are searching an array for a value, and the value is not in the array. What do you return for an index into the array? People often use -1 as the "not found" value. But then what happens when the -1 value is not noticed? It winds up corrupting further attempts to use it. The problem is that integers do not have a NaN value to use for this.

You return (value, found), or (value,error) or Result.

>NaN has value beyond that. Suppose you have an array of sensors. One of those sensors goes bad (like they always do). What value to you use for the bad sensor? NaN. Then, when the data is crunched, if the result is NaN, you know that your result comes from bad data. Compare with setting the bad input to 0.0. You never know how that affects your results.

You return error and handle the error. You want to know sensor is wonky or returns bad data.

Also you technically should use signalling NaN for "this is error" and quiet NaN for "this just impossible math result", which makes it even more error prone. Just return fucking error if it is a function.

Sure, useful for expressions but the handling should be there and then, and if function can have error it should return it explicitly as error, else you have different error handling for different types of functions.

> This is why D (in one of its more controversial choices) sets uninitialized floating point values to NaN rather than the more conventional choice of 0.0.

I'd like to see how much of the code actually uses that as a feature and not just sets it to 0.0 (or initializes it right away)

Re: Examples of floating point problems

#122
post #80

Earlier quoted context omitted.

> An option/maybe type would solve this much better. NaN's are already an option type, although implemented in hardware. The checking comes for free. > An exception would be much better You can configure the FPU to cause an Invalid Operation Exception, but I personally don't find that attractive.

The missing bit is language tooling. The regular floating point API exposed by most languages don’t force handling of NaNs. The benefit of the option type is not necessarily just the extra value, but also the fact that the API that forces you to handle the None value. It’s the difference between null and Option. Even if the API was better, I think there’s value in expressing it as Option which compiles down to using…

Yeah. You should be very explicit about it. Certainly not treat it like, “ooh, here are some free bits that I can use to tag things in ad hoc ways (like -1 for missing index)”.

https://internals.rust-lang.org/t/pre-rfc-nonnan-type/8418

Re: Examples of floating point problems

#123

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> The result of any operation on a NaN is a NaN. That's not true! maxNum(nan, x) = x.

It's complicated.

https://754r.ucbtest.org/background/minNum_maxNum_Removal_De...

Re: Examples of floating point problems

#124
post #75
post #71

Earlier quoted context omitted.

I don't find this convincing. > What do you return for an index into the array? An option/maybe type would solve this much better. > Yes, I know, it can be clumsy to trace it back to its source An exception would be much better, alerting you to the exact spot where the problem occurred.

Exceptions are actually part of floats, they're called "signalling nans". So technically Python is correct when it decided that 0.0/0.0 should raise an exception instead of just quietly returning NaN. Raising an exception is a standards-conforming option. https://stackoverflow.com/questions/18118408/what-is-the-dif...

technically I guess it should return sNAN (so app can check for it if it want to handle it differently) and raise exception if sNaN is used in (non-comparison) operation

Re: Examples of floating point problems

#125
post #120
post #98

Earlier quoted context omitted.

> whether we can rely on (say) the operation 2.0 + 3.0 giving exactly 5.0 (we can!) Can we rely upon 2.3 + 2.3 giving exactly 4.6 though? Can we rely upon LargeInt.0 + LargeInt.0 giving exactly 2xLargeInt.0 for all integers?

That's exactly my point: when you internalize the diagram, you'll be able to reason confidently about what happens: • In the case of "2.3 + 2.3", each "2.3" is “snapped” to the nearest representable value (green line in the diagram), then their sum snapped to the nearest green line. In this case, because the two summands are equal and we're using binary floating-point, the result will also be a green line. If you kne…

> If you knew more about the details of binary64 aka float64, you could confidently say that "2.3" means 2.29999995231628417969 […]"

Then I think writing

    let f = 2.3;
Should be a compile error. The compiler should force you to write the “snapped” value in order to not mislead. :)

Re: Examples of floating point problems

#126

Earlier quoted context omitted.

> Which languages do not have a function to test for NaN? Both C and C++. > This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags. That's why I said in the second part that we need to do away with them.

> Both C and C++. Seems C++ only added it in C++11. Surprising. > That's why I said in the second part that we need to do away with them. Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down. I'd wager that for the majority of programs written, the fast math is as goo…

> Seems C++ only added it in C++11

I added it to Digital Mars C and C++ in the early 1990s. Also full NaN support in all the math.h functions. AFAIK it was the first C compiler to do it.

It was based on a specification worked out by NCEG (Numerical C Extensions Group), a long forgotten group that tried to modernize C numerics support.

> the fast math is as good as the accurate math

I was more interested in correct results than fast wrong answers.

Re: Examples of floating point problems

#127

Earlier quoted context omitted.

> Both C and C++. Seems C++ only added it in C++11. Surprising. > That's why I said in the second part that we need to do away with them. Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down. I'd wager that for the majority of programs written, the fast math is as goo…

> Seems C++ only added it in C++11 I added it to Digital Mars C and C++ in the early 1990s. Also full NaN support in all the math.h functions. AFAIK it was the first C compiler to do it. It was based on a specification worked out by NCEG (Numerical C Extensions Group), a long forgotten group that tried to modernize C numerics support. > the fast math is as good as the accurate math I was more interested in correct re…

Yes, but for most C++ applications, they prefer the faster wrong answers, because the wrong answers are not wrong enough to cause any bugs.

Re: Examples of floating point problems

#128

Earlier quoted context omitted.

> Both C and C++. Seems C++ only added it in C++11. Surprising. > That's why I said in the second part that we need to do away with them. Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down. I'd wager that for the majority of programs written, the fast math is as goo…

> I'd wager that for the majority of programs written, the fast math is as good as the accurate math. I'd take that wager. I spent 13 years working on video games and video game technology, a domain where floating point performance is critical, and by and large we never used fast-math because of the problems it created for us.

Happy to take that wager, because the majority of programs written are not video games related :-)

Re: Examples of floating point problems

#129

Earlier quoted context omitted.

isnan is a macro, not a function

isnan is a macro in C, function in C++. But either way, it’s an abstraction that provides a standard test for NaNs in those languages.

I believe the reason he is pointing out this distinction is that if it is a macro, it can be optimized out causing bugs.

Re: Examples of floating point problems

#130

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. Really though, I'm a fan, I just think that we need be…

> I think the concept of NaNs are sound, but I think relying on them is fraught with peril

Well, allegedly in D at least https://www.reddit.com/r/rust/comments/a1w75c/the_bug_i_did_...

Post reply on HN