Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Floating Point Visually Explained (2017)

fabiensanglard.net

Post reply on HN
Print view View original

31–40 of 101 posts

Re: Floating Point Visually Explained (2017)

#31

Post by cambalache »

adrianmonk wrote: post #26

Here's how I like to think of it. Floating point numbers are just fractions but with one extra condition: the denominator is a power of two. The normal rules of fractions apply. If you want to add them, you have to make sure the denominators match, which would involve scaling the numerators too. Just like fractions, there are multiple ways of writing the same value. 3/2 is the same as 6/4. You can't write 1/3 exactly…

> Just like fractions, there are multiple ways of writing the same value. 3/2 is the same as 6/4.

This is not correct. In binary you only have 2 choices {0,1} for the numerator, so 3/2 is represented 1.1 and 6/4 can only be represented as 1.1 (feel free to add the completing 0s)

Re: Floating Point Visually Explained (2017)

#32

Post by adrianmonk »

cambalache wrote: post #31
adrianmonk wrote: post #26

Here's how I like to think of it. Floating point numbers are just fractions but with one extra condition: the denominator is a power of two. The normal rules of fractions apply. If you want to add them, you have to make sure the denominators match, which would involve scaling the numerators too. Just like fractions, there are multiple ways of writing the same value. 3/2 is the same as 6/4. You can't write 1/3 exactly…

> Just like fractions, there are multiple ways of writing the same value. 3/2 is the same as 6/4. This is not correct. In binary you only have 2 choices {0,1} for the numerator, so 3/2 is represented 1.1 and 6/4 can only be represented as 1.1 (feel free to add the completing 0s)

The numerator is a binary integer, not a bit (and not the part of the number before the "decimal" point).

3 in binary is 11. 6 in binary is 110.

3/2 (decimal) is 11/10 (binary) and 6/4 (decimal) is 110/100 (binary).

Of course the denominators always have trailing zeros, so can be (and are) represented more compactly.

Re: Floating Point Visually Explained (2017)

#33

Post by jimmaswell »

I remember wondering at first how you know this won't have multiple representations and cover every number - it's because the mantissa can't reach 2, and if it was 2 it would be the same as adding to the exponent, so you get the full range between any two exponents.

Re: Floating Point Visually Explained (2017)

#34

Post by piadodjanho »

jimmaswell wrote: post #33

I remember wondering at first how you know this won't have multiple representations and cover every number - it's because the mantissa can't reach 2, and if it was 2 it would be the same as adding to the exponent, so you get the full range between any two exponents.

Ackchyually...

The IEEE-754 has a lot of redundant representation. Not where you would expect though.

Caveat: Those features are invaluable for some niche applications, but not for the average joe.

To start. Every IEEE-754 float has two zero representation: one for positive zero and another negative negative zero (sic).

The special numbers are another source of redundancy. The the double format, have about 9,007,199,254,740,992 different combination to encode three different states that a production ready software shouldn't reach: NaN, +inf and -inf.

Other than the redundancy, the double have many rarely used combination. For instance, the subnormals representation. Unless you are compiling your program with -O0 or with some exotic compiler, they are disabled by default. One subnormal operation can take over a hundread of cycles to complete. Therefore, more 9,007,199,254,740,992 wasted combination.

If that wasn't bad enough, since the magnitude of the numbers follows a normal distribution (someone whose name I forgot's law), the most significant bits of the exponent field are very rarely used. The IEEE-754 encoding is suboptimal.

The posit floating point address all those issues. It uses an tapered encoding for the exponent field.

Re: Floating Point Visually Explained (2017)

#35

Post by romwell »

randyrand wrote: post #28
augustt wrote: post #13

Didn't everyone learn scientific notation in high school? It's pretty much exactly that, and you could put the coefficient/exponent into whatever bit pattern you'd like.

It's not exactly scientific notation because a leading bit is assumed for normalized numbers. The window and offset explanation naturally accounts for the leading bit, whereas the scientific notation explanation needs further explanation to explain how the leading bit works. In short, 10^2 * .001 is not allowed in floating point. You can't have a mantissa that starts with 0. The leading bit means all mantissas must b…

I think this is worth saying about the leading bit (why 0.001 is not a valid mantissa):

In binary, we always know the first non-zero digit of any number - so there's no need to write it down. We know it's 1 because, well, binary.

So we don't waste space, and only write down all the other digits, and then use the exponent to put the 'point' into the right place.

We save a bit of space doing this.

Re: Floating Point Visually Explained (2017)

#36

Post by GuB-42 »

augustt wrote: post #13

Didn't everyone learn scientific notation in high school? It's pretty much exactly that, and you could put the coefficient/exponent into whatever bit pattern you'd like.

It is scientific notation with a twist. The twist is the implicit 1. in the mantissa.

There is no easy way to make it work in decimal. So you need to learn how fractional parts work in binary, then move up to scientific notation in binary. Then you probably noticed that all numbers start with 1, so you can knock it off to save space. Oh, and add a special case for zero, because it is the only number that doesn't start with 1, and if you are feeling adventurous, there are subnormal numbers.

That's why I find the explanation in the article absolutely brilliant. By treating the exponent as a window, not only you don't need to bother with scientific notation in binary, but zero and subnormal numbers fit very nicely.

In fact, I've been working for years with floating point numbers and feel stupid for not realizing it works just as described in the article. And feeling stupid after reading something is usually a very good sign. I clicked, expected an article I could dismiss in a typical Hacker News fashion, but no, not this time.

Re: Floating Point Visually Explained (2017)

#37

Post by piadodjanho »

enriquto wrote: post #15
seanalltogether wrote: post #5

I always found the wikipedia examples for 16bit floating point helpful since the numbers are smaller. You can really see how the exponent and fraction affect each other in a very simple way. https://en.wikipedia.org/wiki/Half-precision_floating-point_...

The 8-bit floating point is even easier to understand, since you can list them all in a single page, and even visualize the entire addition and multiplication tables.

Also a great format to count how many representation are wasted with redundant representation (ZeroS, NaN, +inf, -inf).

Re: Floating Point Visually Explained (2017)

#38

Post by anamexis »

augustt wrote: post #13

Didn't everyone learn scientific notation in high school? It's pretty much exactly that, and you could put the coefficient/exponent into whatever bit pattern you'd like.

You just repeated the "how everybody hates floating point to be explained to them" section of this article.

Re: Floating Point Visually Explained (2017)

#40

Post by thechao »

piadodjanho wrote: post #34
jimmaswell wrote: post #33

I remember wondering at first how you know this won't have multiple representations and cover every number - it's because the mantissa can't reach 2, and if it was 2 it would be the same as adding to the exponent, so you get the full range between any two exponents.

Ackchyually... The IEEE-754 has a lot of redundant representation. Not where you would expect though. Caveat: Those features are invaluable for some niche applications, but not for the average joe. To start. Every IEEE-754 float has two zero representation: one for positive zero and another negative negative zero (sic). The special numbers are another source of redundancy. The the double format, have about 9,007,199,…

I'm mixed on Gustafson's posit stuff. For me, the only thing I'd change for fp would be:

1. -0 now encodes NAN.

2. +inf/-inf are all Fs with sign: 0x7FFFFFFF, 0xFFFFFFFF.

3. 0 is the only denorm.

Which does four good things:

1. Gets rid of the utter insanity which is -0.

2. Gets rid of all the redundant NANs.

3. Makes INF "look like" INF.

4. Gets rid of "hard" mixed denorm/norm math.

And one seriously bad thing:

1. Lose a bunch of underflow values in the denorm range.

However, as to the latter: who the fuck cares! Getting down to that range using anything other than divide-by-two completely trashes the error rate anyways, so why bother?

The rest of Gustafson's stuff always sounds like crazy-people talk, to me.

Post reply on HN