Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Examples of floating point problems

jvns.ca

Post reply on HN
Print view View original

101–110 of 179 posts

Re: Examples of floating point problems

#101

Post by BeetleB »

maximilianburke wrote: post #83
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. Really though, I'm a fan, I just think that we need be…

> made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)")

Which languages do not have a function to test for NaN?

> and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test.

This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags.

Re: Examples of floating point problems

#102

Post by jcranmer »

WalterBright wrote: post #82
jordigh wrote: post #75

Earlier quoted context omitted.

Exceptions are actually part of floats, they're called "signalling nans". So technically Python is correct when it decided that 0.0/0.0 should raise an exception instead of just quietly returning NaN. Raising an exception is a standards-conforming option. https://stackoverflow.com/questions/18118408/what-is-the-dif...

In practice, I've found signalling NaNs to be completely unworkable and gave up on them. The trouble is they eagerly convert to quiet NaNs, too eagerly.

I am firmly in the belief that sNaNs were a mistake in IEEE 754, and all they really serve to do is to create hard trivia questions for compiler writers.

Re: Examples of floating point problems

#103

Post by BeetleB »

inetknght wrote: post #85
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This means that NaNs propagate from the source of the original NaN to the final printed result. An exception would be better. Then you immediately get at the first problem instead of having to track down the lifetime of the observed problem to find the first problem.

> An exception would be better.

It depends on the context. Sometimes NaNs are expected and to be ignored. Sometimes they signal a problem.

Re: Examples of floating point problems

#104

Post by maximilianburke »

BeetleB wrote: post #101
maximilianburke wrote: post #83

Earlier quoted context omitted.

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. Really though, I'm a fan, I just think that we need be…

> made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)") Which languages do not have a function to test for NaN? > and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. This is not unique to NaNs. There are plenty of potential floating point problems if…

> Which languages do not have a function to test for NaN?

Both C and C++.

> This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags.

That's why I said in the second part that we need to do away with them.

Re: Examples of floating point problems

#105

Post by BeetleB »

> example 4: different languages sometimes do the same floating point calculation differently

It's worse than that: You can use the same language, compiler, library, machine, and still get different results if your OS is different.

I forget all the details, but it boils down to how intermediate results are handled. When you compute certain functions, there are several intermediate calculations before it spits out the result. You get more accuracy if you allow those intermediate calculations to happen in a higher precision format (e.g. you're computing in 32 bits, so it will compute the intermediate values in 64 bits). But that is also slower.

OS's make a "default" choice. I think Linux defaults to slower, but more accurate, and BSD defaults to faster, but less accurate.

There may be flags you can set to force one configuration regardless of the OS, but you shouldn't assume your libraries do that.

> In principle you might think that different implementations should work the same way because of the IEEE 754 standard for floating point, but here are a couple of caveats that were mentioned:

> math operations in libc (like sin/log) behave differently in different implementations. So code using glibc could give you different results than code using musl

IEEE 754 doesn't mandate a certain level of accuracy for transcendental functions like sin/log. You shouldn't expect different libraries to give you the same value. If you're doing 64 bit calculations, I would imagine most math libraries will give results accurate enough for 99.99% of math applications, even if only the first 45 bits are correct (and this would be considered "very inaccurate" by FP standards).

Re: Examples of floating point problems

#106

Post by subharmonicon »

maximilianburke wrote: post #104
BeetleB wrote: post #101

Earlier quoted context omitted.

> made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)") Which languages do not have a function to test for NaN? > and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. This is not unique to NaNs. There are plenty of potential floating point problems if…

> Which languages do not have a function to test for NaN? Both C and C++. > This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags. That's why I said in the second part that we need to do away with them.

isnan() has been around since C99.

Re: Examples of floating point problems

#107

Post by [deleted] »

maximilianburke wrote: post #104
BeetleB wrote: post #101

Earlier quoted context omitted.

> made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)") Which languages do not have a function to test for NaN? > and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. This is not unique to NaNs. There are plenty of potential floating point problems if…

> Which languages do not have a function to test for NaN? Both C and C++. > This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags. That's why I said in the second part that we need to do away with them.

[deleted]

Re: Examples of floating point problems

#108

Post by sacrosancty »

svat wrote: post #58
sacrosancty wrote: post #50

Earlier quoted context omitted.

If you have even less time, just think of them as representing physical measurements made with practical instruments and the math done with analog equipment. The common cause of floating point problems is usually treating them as a mathematical ideal. The quirks appear at the extremes when you try to to un-physical things with them. You can't measure exactly 0 V with a voltmeter, or use an instrument for measuring th…

Thanks, I actually edited my post (made the second paragraph longer) after seeing your comment. The "physical" / "analog" idea does help in one direction (prevents us from relying on floating-point numbers in unsafe ways) but I think it brings us too close to the "superstition" end of the spectrum, where we start to think that floating-point operations are non-deterministic, start doubting whether we can rely on (say…

[dead]

Re: Examples of floating point problems

#109

Post by BeetleB »

maximilianburke wrote: post #104
BeetleB wrote: post #101

Earlier quoted context omitted.

> made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)") Which languages do not have a function to test for NaN? > and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. This is not unique to NaNs. There are plenty of potential floating point problems if…

> Which languages do not have a function to test for NaN? Both C and C++. > This is not unique to NaNs. There are plenty of potential floating point problems if you enable those flags. That's why I said in the second part that we need to do away with them.

> Both C and C++.

Seems C++ only added it in C++11. Surprising.

> That's why I said in the second part that we need to do away with them.

Do away entirely with fast math calculations? That would be horrible. They exist for a very good reason: Some applications are too slow without them. Enabling subnormal numbers can really slow things down.

I'd wager that for the majority of programs written, the fast math is as good as the accurate math.

Post reply on HN