Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Floating Point Visually Explained (2017)

fabiensanglard.net

Post reply on HN
Print view View original

11–20 of 101 posts

Re: Floating Point Visually Explained (2017)

#11

Post by saagarjha »

wenc wrote: post #7
biddlesby wrote: post #3

I saw a very simple visualisation of floating point by simply plotting points along the number line and zooming out. That gets the idea across very quickly!

Yes, I came across something like that in my numerical computation course. This was for me the most compelling visualization of what a floating point number was. The floating number line is actually an (unevenly spaced in reals, uniformly spaced in binary) discrete number line that was used to approximate continuous infinite real numbers, in finite precision e.g. Reals 0 ______________________________________________…

A more accurate representation might be

  0.... . . . .  .  .  .  .    .    .    .    1
(Hacker News eats consecutive spaces, it seems like :/)

Re: Floating Point Visually Explained (2017)

#12

Post by saagarjha »

zozbot234 wrote: post #9

I believe that this "window and offset" intuition, while indeed true and useful in the radix-2 ("binary") case, does not cleanly extend to the general case where no hidden bit is used even for non-"subnormal" numbers, and some numbers may thus have multiple representations. This shows up perhaps most clearly in the case of decimal floating point, but ISTR that a non-2 radix was also used in some mainframes.

Subnormal numbers are a blemish on the otherwise beautiful IEEE 754 specification :(

Re: Floating Point Visually Explained (2017)

#15

Post by enriquto »

seanalltogether wrote: post #5

I always found the wikipedia examples for 16bit floating point helpful since the numbers are smaller. You can really see how the exponent and fraction affect each other in a very simple way. https://en.wikipedia.org/wiki/Half-precision_floating-point_...

The 8-bit floating point is even easier to understand, since you can list them all in a single page, and even visualize the entire addition and multiplication tables.

Re: Floating Point Visually Explained (2017)

#16

Post by Someone »

bogomipz wrote: post #4
commandlinefan wrote: post #2

Wow, that's a much easier way to convert from decimal to floating point than I had ever seen. He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order.

>"He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order." Could you elaborate on this? Maybe an example? This sounds interesting but I'm failing to grasp it.

https://wiki.c2.com/?IeeeSevenFiftyFour:

“IEEE 754 […] Has the interesting and useful property that two's complement comparisons of the underlying bit pattern of any two IEEE 754 numbers will have the same result as comparing the numbers that are represented”

That means that, if you interpret the bits of a float/double as an int32/int64, increase that integer by one, and then interpret the bits of the result as a float/double, you get the smallest float/double that’s larger than what you started with (with exceptions for NaN, infinities, +/- zero, and, possibly, some categories I forget)

That can be useful if you want to iterate over all floats between 0.0 and 1.0, for example, but may not be that efficient on some modern hardware, where moving data from the “float side” of the CPU to the “integer side” is expensive)

Re: Floating Point Visually Explained (2017)

#17

Post by SupriseAnxiety »

This right here is the only way I pretty much figured out just now a more precise understanding of the weird patterns going on in my head. I crave precision and this explained more than a whole year covering Basic C. Thank you so much! My head doesn’t have to hurt over that anymore. I never understood floats either but this representation truly helps clear some fog!!

Re: Floating Point Visually Explained (2017)

#18

Post by gopalv »

seanalltogether wrote: post #5

I always found the wikipedia examples for 16bit floating point helpful since the numbers are smaller. You can really see how the exponent and fraction affect each other in a very simple way. https://en.wikipedia.org/wiki/Half-precision_floating-point_...

BFloat16[1] is an interesting tweak to the original, but built with more modern requirements for ML in mind.

Particularly because it is easier to think of 32 bit IEEE-754 to BFloat16, but with fewer bits of think about (& possibly you can enumerate the entire range in a laptop to test something like "will this function work for all values?").

[1] - https://en.wikipedia.org/wiki/Bfloat16_floating-point_format...

Re: Floating Point Visually Explained (2017)

#20

Post by saagarjha »

Someone wrote: post #16
bogomipz wrote: post #4

Earlier quoted context omitted.

>"He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order." Could you elaborate on this? Maybe an example? This sounds interesting but I'm failing to grasp it.

https://wiki.c2.com/?IeeeSevenFiftyFour : “IEEE 754 […] Has the interesting and useful property that two's complement comparisons of the underlying bit pattern of any two IEEE 754 numbers will have the same result as comparing the numbers that are represented” That means that, if you interpret the bits of a float/double as an int32/int64, increase that integer by one, and then interpret the bits of the result as a fl…

This has more to do with the mantissa than the biased exponent.
Post reply on HN