Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Examples of floating point problems

jvns.ca

Post reply on HN
Print view View original

81–90 of 179 posts

Re: Examples of floating point problems

#81

Post by jcparkyn »

WalterBright wrote: post #74
pwpwp wrote: post #71

Earlier quoted context omitted.

I don't find this convincing. > What do you return for an index into the array? An option/maybe type would solve this much better. > Yes, I know, it can be clumsy to trace it back to its source An exception would be much better, alerting you to the exact spot where the problem occurred.

> An option/maybe type would solve this much better. NaN's are already an option type, although implemented in hardware. The checking comes for free. > An exception would be much better You can configure the FPU to cause an Invalid Operation Exception, but I personally don't find that attractive.

> NaN's are already an option type, although implemented in hardware

The compromise with this is that it makes it impossible to represent a non-optional float, which leads to the same issues as null pointers in c++/java/etc.

The impacts of NaN are almost certainly not as bad (in aggregate) as `null`, but it'd still be nice if more languages had ways to guarantee that certain numbers aren't NaN (e.g. with a richer set of number types).

Re: Examples of floating point problems

#82

Post by WalterBright »

jordigh wrote: post #75
pwpwp wrote: post #71

Earlier quoted context omitted.

I don't find this convincing. > What do you return for an index into the array? An option/maybe type would solve this much better. > Yes, I know, it can be clumsy to trace it back to its source An exception would be much better, alerting you to the exact spot where the problem occurred.

Exceptions are actually part of floats, they're called "signalling nans". So technically Python is correct when it decided that 0.0/0.0 should raise an exception instead of just quietly returning NaN. Raising an exception is a standards-conforming option. https://stackoverflow.com/questions/18118408/what-is-the-dif...

In practice, I've found signalling NaNs to be completely unworkable and gave up on them. The trouble is they eagerly convert to quiet NaNs, too eagerly.

Re: Examples of floating point problems

#83

Post by maximilianburke »

WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test.

Really though, I'm a fan, I just think that we need better means for checking them in legacy languages and we need to entirely do away with "fast math" optimizations.

Re: Examples of floating point problems

#84

Post by WalterBright »

maximilianburke wrote: post #83
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. Really though, I'm a fan, I just think that we need be…

I call them "buggy math" optimizations. The dmd D compiler does not have a switch to enable buggy math.

Re: Examples of floating point problems

#85

Post by inetknght »

WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This means that NaNs propagate from the source of the original NaN to the final printed result.

An exception would be better. Then you immediately get at the first problem instead of having to track down the lifetime of the observed problem to find the first problem.

Re: Examples of floating point problems

#86

Post by omginternets »

WalterBright wrote: post #74
pwpwp wrote: post #71

Earlier quoted context omitted.

I don't find this convincing. > What do you return for an index into the array? An option/maybe type would solve this much better. > Yes, I know, it can be clumsy to trace it back to its source An exception would be much better, alerting you to the exact spot where the problem occurred.

> An option/maybe type would solve this much better. NaN's are already an option type, although implemented in hardware. The checking comes for free. > An exception would be much better You can configure the FPU to cause an Invalid Operation Exception, but I personally don't find that attractive.

As far as I'm aware, there's no equivalent to a stack trace with NaN, so finding the origin of a NaN can be extremely tedious.

Re: Examples of floating point problems

#87

Post by zokier »

jordigh wrote: post #67
zokier wrote: post #61

Earlier quoted context omitted.

I don't think Pythons Decimal is ieee754, instead its some sort of arbitrary precision thingy. GCC has builtin support for decimal floats: https://gcc.gnu.org/onlinedocs/gcc/Decimal-Float.html There are also library implementations floating around, some of them are mentioned in this thread: https://discourse.llvm.org/t/rfc-decimal-floating-point-supp... decnumber has also rust wrappers if you are so inclined

Python's decimal absolutely is IEEE 754 (well, based on the older standard, which has now been absorbed into IEEE 754): https://github.com/python/cpython/blob/main/Lib/_pydecimal.p... Cool, didn't know that gcc had built-in support. But is it really as incomplete as it says there?

Huh, I didn't know it was that close, I'll grant that. But I'd say still no cigar.

One of the most elementary requirements of IEEE754 is:

> A programming environment conforms to this standard, in a particular radix, by implementing one or more of the basic formats of that radix as both a supported arithmetic format and a supported interchange format.

(Section 3.1.2)

While you could argue that you may configure Decimals context parameters to match those of some IEEE754 format and thus claim conformance as arithmetic format, Python has absolutely no support for the specified interchange formats.

To be honest, seeing this I'm bit befuddled on why closer conformance with IEEE754 is not sought. Quick search found e.g. this issue report on adding IEEE754 parametrized context, which is a trivial patch, and it has been just sitting there for 10 years: https://github.com/python/cpython/issues/53032

Adding code to import/export BID/DPD formats, while maybe not as trivial, seems still comparatively small task and would improve interoperability significantly imho.

Re: Examples of floating point problems

#88

Post by insulanus »

inetknght wrote: post #85
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This means that NaNs propagate from the source of the original NaN to the final printed result. An exception would be better. Then you immediately get at the first problem instead of having to track down the lifetime of the observed problem to find the first problem.

Definitely. Unfortunately, Language implementations that guaranteed exceptions were not in wide use at the time. Also, to have a chance at being implemented on more than one CPU, it had to work in C and assembly.

Re: Examples of floating point problems

#89

Post by WalterBright »

omginternets wrote: post #86
WalterBright wrote: post #74

Earlier quoted context omitted.

> An option/maybe type would solve this much better. NaN's are already an option type, although implemented in hardware. The checking comes for free. > An exception would be much better You can configure the FPU to cause an Invalid Operation Exception, but I personally don't find that attractive.

As far as I'm aware, there's no equivalent to a stack trace with NaN, so finding the origin of a NaN can be extremely tedious.

I've never found it to be particularly difficult.

The extremely difficult problems to find are uninitialized data and threading bugs, mainly because they appear and disappear.

Re: Examples of floating point problems

#90

Post by maxfurman »

jordigh wrote: post #72
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> What's the result of 1.0/0.0? It's not a number, so it's a NaN It's not often that I get to correct Mr D himself, but 1.0/0.0 is...

What is 1.0/0.0 in D? A DivideByZero exception of some sort?
Post reply on HN