Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perspectives on Floating Point

eigentales.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Perspectives on Floating Point

#11

Post by hollerith »

exmadscientist wrote: post #8
hollerith wrote: post #6

Earlier quoted context omitted.

>better optimizations as the not-normal values throw a wrench into optimizing math. I'm not an expert on architecture, but I would've guessed that the need to branch to process the error would be the wrench and that the use of NaN and the infinities allow better optimizations.

IEEE-754 NaN and Infinity have nothing to do with optimization. They come straight from math: * What is +1/0? It has to be +Infinity -- nothing else will do. * What is -1/0? It has to be -Infinity -- nothing else will do. * What is 0/0? There's no way to tell from the information we've got -- it's undefined: Not A Number. (However, should 0/0 come up as a result of taking the quotient of two functions that happen to…

NaN and Infinity have nothing to do with optimization only if your mental model of CPUs is extremely simplistic.

Re: Perspectives on Floating Point

#12

Post by jeezfrk »

yxhuvud wrote: post #4

One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…

It would be quite useful to protect other code from that.

It could easily be a compiler-based protection: any opcode that can generate it from two/one normal input must be checked for normal results.

This is similar to how C++ gives few guarantees but "new" and "this" never give null or unallocated values.

Re: Perspectives on Floating Point

#15

Post by dunham »

kolbusa wrote: post #13

Not sure why the article does not reference the following paper which is a must read for anyone working with floating point: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h... (original: https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf ).

I also recently saw this paper on the difficulty of solving the quadratic equation with floating point numbers:

https://cnrs.hal.science/hal-04116310/document

And also Gerald Sussman saying:

> The only thing that scares me in programming is floating point.

https://youtu.be/Tdwr9tweTDE?t=1145

Re: Perspectives on Floating Point

#16

Post by fweimer »

exmadscientist wrote: post #8
hollerith wrote: post #6

Earlier quoted context omitted.

>better optimizations as the not-normal values throw a wrench into optimizing math. I'm not an expert on architecture, but I would've guessed that the need to branch to process the error would be the wrench and that the use of NaN and the infinities allow better optimizations.

IEEE-754 NaN and Infinity have nothing to do with optimization. They come straight from math: * What is +1/0? It has to be +Infinity -- nothing else will do. * What is -1/0? It has to be -Infinity -- nothing else will do. * What is 0/0? There's no way to tell from the information we've got -- it's undefined: Not A Number. (However, should 0/0 come up as a result of taking the quotient of two functions that happen to…

I'm pretty sure IEEE 754 covers trapping floating point math. It's not just about in-band signaling. It's unfortunate that the standard is proprietary, so we can't easily reference it to figure out what it says.

Anyway, most CPUs support a trapping mode, after all. Here's an example with glibc:

    #define _GNU_SOURCE
    #include 
    
    volatile double x = 1.0;
    volatile double y;
    volatile double quotient;
    
    int
    main(void)
    {
      feenableexcept(FE_DIVBYZERO);
      quotient = x / y;
    }
As far as I understand it, overall support for trapping math is poor because not much code is trapping-aware. It would be quite annoying if JSON parsing results in SIGFPE due to an Inexect trap, for example.

Re: Perspectives on Floating Point

#17

Post by yxhuvud »

hollerith wrote: post #6
yxhuvud wrote: post #4

One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…

>better optimizations as the not-normal values throw a wrench into optimizing math. I'm not an expert on architecture, but I would've guessed that the need to branch to process the error would be the wrench and that the use of NaN and the infinities allow better optimizations.

It is not about hardware, it is about some of the optimizations you can enable when assuming they can't happen. There are some mathematical identities that only work if you can assume they can't happen.

Re: Perspectives on Floating Point

#18

Post by dahart »

rhythane wrote: post #3

A really interesting review. The idea of relative error makes sense in most cases, but when we need to do subtraction and difference matters, maybe absolute error is actually better.

Subtraction is the very case where relative error might matter most, and error relative to magnitude of the original unsubtracted numbers causes the most surprises.

Absolute error has useful applications, without any doubt, and regardless of arithmetic operation, but it probably doesn’t make sense to say it’s “better” without a specific problem in front of us, and without specific goals and priorities. Error tolerance is always up to the user.

Re: Perspectives on Floating Point

#19

Post by dahart »

yxhuvud wrote: post #4

One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…

What better error handling can one do without reserved special numbers?

Many (most?) compilers already offer a “fast math” flag that foregoes denormals. You don’t need to remove denormals from the format in order to avoid them.

Having a separate type without specials would probably cause unending confusion and increase bugs. It’s already the case that only people who care about the specials handle them, but it sounds like there are more of those people than you think, if you think only mathematicians care about them.

Re: Perspectives on Floating Point

#20

Post by SideQuark »

otabdeveloper4 wrote: post #10
yxhuvud wrote: post #4

One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…

"Mathematicians" don't need NaN's and Inf's. These things come straight from overflow and underflow behaviour of floating point bit representations. I'd guess some sort of IEEE flag exists to emit an interrupt if over/underflow happens, but nobody really wants those.

They don’t come solely from over/underflow. The do come from math. Sort of neg, 1/0, and plenty of other mathematical operations result in these, no matter how you want to store numbers.
Post reply on HN