This is beautifully written. Bravo to the author.
Floating Point Visually Explained (2017)
81–90 of 101 posts
Re: Floating Point Visually Explained (2017)
#82Earlier quoted context omitted.
No, it has to do with bothequally. Changing a 0 bit to a 1 in an integer increases its value, and by the rule I gave, should also increase the value of the floating point number with the same bit pattern. It doesn’t matter whether that flipped bit is in the mantissa or the exponent. That requires the use of the biased number in the exponent. In particular, the “all zeroes” bit pattern for the exponent must be the rep…
This is a different claim than the one above, which was about incrementing the least significant bit.
But it trivially follows from it. If operation foo (increasing x by one) makes something larger, repeated operation of foo (increasing x by 2^n) makes it larger, too.
Re: Floating Point Visually Explained (2017)
#83Another visualization that blew my mind; I learnt this from a presentation that was about doing floats with FPGAs:
Think 32-bit floats as a 256-bit buffer interpreted as a fixed precision number, with the decimal point straight in the middle, but with the limitation that you can set only a continuous 24-bit window on that buffer to non-zeros.
Then, the 8 bits of the exponent determine where in the bit buffer that window points to, and the 23 (+1) bits are the contents of the window.
Re: Floating Point Visually Explained (2017)
#84I wonder why the significand is represented as a > 1 number. It complicates representing zero (making it a special case not covered by the article of defining exponent=0, mantissa=0 to mean zero). Is it for the purpose of simplifying arithmetic operations or is it to minimize the number of redundant representations of zero?
Re: Floating Point Visually Explained (2017)
#85Earlier quoted context omitted.
>"He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order." Could you elaborate on this? Maybe an example? This sounds interesting but I'm failing to grasp it.
https://wiki.c2.com/?IeeeSevenFiftyFour : “IEEE 754 […] Has the interesting and useful property that two's complement comparisons of the underlying bit pattern of any two IEEE 754 numbers will have the same result as comparing the numbers that are represented” That means that, if you interpret the bits of a float/double as an int32/int64, increase that integer by one, and then interpret the bits of the result as a fl…
[deleted]
Re: Floating Point Visually Explained (2017)
#86Earlier quoted context omitted.
On the other hand, if you don't have subnormals, then you have the funny property that subtracting two inequal numbers would yield 0. This never happens with subnormals because they allow representing numbers closer to zero, below the smallest representable exponent. The numerical stability of some algorithms crucially relies on subnormals.
I do not understand your statement. I can come up with two possible interpretations, but neither seem true to me. Can you provide an example of what you mean? I am also interested in your statement that certain algorithms depend on subnormals as defined in the floating point spec. Can you provide an example of such an algorithm? I can intuit how it might be desirable to have a single "too small/epsilon" value, but I…
If x and y are finite floats (not NaN or ±Infinity), with x != y, then x - y != 0. This is true for any pair of floats only because of denormal numbers. If there were no denormals (or denormals were flushed to zero), then if |x - y| < 2^-127 then x - y would become zero.
Re: Floating Point Visually Explained (2017)
#87Excellent article! Just one thing surprised me:
> While I was writing a book about Wolfenstein 3D[1], I wanted to vividly demonstrate how much of a handicap it was to work without floating point
I would've expected fixed point to work fine for games, because that's a domain where you know the data, and in particular the dynamic range, in advance, so the 'automatically adjust to whatever dynamic range happens to be in the data' feature of floating point isn't needed. What am I missing? (If the answer is 'it would take too long to explain here, but he does actually explain it in the book', I'm prepared to accept that.)
Re: Floating Point Visually Explained (2017)
#88Earlier quoted context omitted.
>"He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order." Could you elaborate on this? Maybe an example? This sounds interesting but I'm failing to grasp it.
Sure, I had to wrestle with this a bit myself: to make it easier, imagine a 16-bit floating point format (P&H call this the "Nvidia format", but I can't find that documented anywhere but there): 1-bit sign, 5-bit (biased) exponent, 10-bit mantissa. One thing that TFA leaves out about floating point mantissas is that there's an implicit leading 1, so a 10-bit mantissa of 1111100000 would be interpreted as (binary) 1.1…
> imagine a 16-bit floating point format (P&H call this the "Nvidia format", but I can't find that documented anywhere but there): 1-bit sign, 5-bit (biased) exponent, 10-bit mantissa
According to Wikipedia (https://en.wikipedia.org/wiki/Half-precision_floating-point_...), this is the IEEE 754 standard binary16 format.
Re: Floating Point Visually Explained (2017)
#89Earlier quoted context omitted.
Also a great format to count how many representation are wasted with redundant representation (ZeroS, NaN, +inf, -inf).
What is redundant about that? It "wastes" a completely negligible part of the representation space, and the consistency gains are enormous.
When talking about tiny 8-bit floats, it does waste a lot: if your exponent is only 3 bits, you've "wasted" 1/8 of all 256 possible values, which is a lot. With normal-sized floats, it's much less of an issue: 1/256 of the billions of possible 32-bit values, and 1/2048 of all possible 64-bit values.
(Also, the real "waste" is only on the multiple NaN values, since the zeros always "waste" only a single value for the "negative zero", and the infinities always "waste" only two values; AFAIK, both negative zero and the infinities are necessary for stability of some calculations.)
Re: Floating Point Visually Explained (2017)
#90I wonder why the significand is represented as a > 1 number. It complicates representing zero (making it a special case not covered by the article of defining exponent=0, mantissa=0 to mean zero). Is it for the purpose of simplifying arithmetic operations or is it to minimize the number of redundant representations of zero?
The leading 1. prefix can be omitted from the binary representation thus saving a bit in the process.