Perspectives on Floating Point
eigentales.com
Perspectives on Floating Point
1–10 of 20 posts
Re: Perspectives on Floating Point
#2Very nice graphics in this.
Re: Perspectives on Floating Point
#3A really interesting review. The idea of relative error makes sense in most cases, but when we need to do subtraction and difference matters, maybe absolute error is actually better.
Re: Perspectives on Floating Point
#4One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizations as the not-normal values throw a wrench into optimizing math.
Re: Perspectives on Floating Point
#5A really interesting review. The idea of relative error makes sense in most cases, but when we need to do subtraction and difference matters, maybe absolute error is actually better.
If you are doing only additive operations, then, yes, absolute error might actually be the best choice. But as soon as multiplications start to show up, they are enough trouble that they tend to dominate the whole error propagation show. Since many real calculations have multiplication in them, you end up having to optimize the whole thing for multiplicative operations, and so we end up just using relative errors everywhere.
You can, of course, do a very specialized optimization for one particular algorithm, but that tends to not be a very good use of time. Usually. (Counterexample: Kahan summation!)
Re: Perspectives on Floating Point
#6One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…
>better optimizations as the not-normal values throw a wrench into optimizing math.
I'm not an expert on architecture, but I would've guessed that the need to branch to process the error would be the wrench and that the use of NaN and the infinities allow better optimizations.
Re: Perspectives on Floating Point
#7One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…
So, kind of, you want every NaN to be a sNan? Along those lines, this SO answer is really interesting: https://stackoverflow.com/a/55648118
Re: Perspectives on Floating Point
#8One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…
>better optimizations as the not-normal values throw a wrench into optimizing math. I'm not an expert on architecture, but I would've guessed that the need to branch to process the error would be the wrench and that the use of NaN and the infinities allow better optimizations.
IEEE-754 NaN and Infinity have nothing to do with optimization. They come straight from math:
* What is +1/0? It has to be +Infinity -- nothing else will do.
* What is -1/0? It has to be -Infinity -- nothing else will do.
* What is 0/0? There's no way to tell from the information we've got -- it's undefined: Not A Number. (However, should 0/0 come up as a result of taking the quotient of two functions that happen to both reach zero at a point, then sometimes the limit of that quotient is meaningful, and might have a numerical result.)
IEEE-754 chose to signal these things in-band, so we get NaN and Infinity to deal with in our floats and doubles.
Re: Perspectives on Floating Point
#9One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…
On the hardware level, your processor will have some specific behavior that it exhibits. This is sometimes configurable, in platform-specific way, and people working in low-level, high-performance numerical code already take advantage of that when it is.
If you more mean the language level, and just having two different ways to represent floating point numbers in your code, it gets a little tricky to reason about because one of those may align with what the processor is doing natively and others won't, and so you'd have a software layer converting the behavior of the platform to what your language promises -- with very high overhead sometimes and almost no overhead other times. That kind of inconsistency isn't needed very often. It can become a real headache for testing and in the wild. It's easier for a language to just say "this is the way we do floats" and you adapt as needed.
So the more typical balance is to deal with that kind of thing at the library level. If you want numbers that behave a certain way, and it's not the way your language models them, you use a library that gives you the kind of numbers you want with the tangible awareness that they're likely "soft" and less efficient in some way.
Re: Perspectives on Floating Point
#10One thing I think would be nice for floating point numbers, is that I'd prefer if there were two separate types - one where NaN and the two infinites are allowed, and one where they are not allowed but instead emit an error. The former would be used by some few mathematicians etc, and the rest of us could use the latter. The upside would be better error handling close to the source of the issue, and better optimizati…
"Mathematicians" don't need NaN's and Inf's.
These things come straight from overflow and underflow behaviour of floating point bit representations. I'd guess some sort of IEEE flag exists to emit an interrupt if over/underflow happens, but nobody really wants those.