31–40 of 58 posts
Floating Point Visually Explained (2017) - https://news.ycombinator.com/item?id=23081924 - May 2020 (96 comments)
Floating Point Visually Explained (2017) - https://news.ycombinator.com/item?id=19084773 - Feb 2019 (17 comments)
Floating Point Visually Explained - https://news.ycombinator.com/item?id=15359574 - Sept 2017 (106 comments)
I was kinda hoping for a visualization of which numbers exists in floating point. While I always new about 0.1 + 0.2 -> 0.30000000000000004 it was still kind of an epiphany realizing that floating point numbers don’t so much have rounding errors as they are simply discreet numbers. You can move from one float to the next, which is a meaningfull operation on discreet numbers like integers, but not continuous numbers l…
What do you mean by continuous? Obviously if you take a number line and remove either the rational or irrational numbers, you will end up with infinitely many holes.
The thing that makes floating point numbers unique is that, for any given representation, there are actually only finitely many. There’s a largest possible value and a smallest possible value and each number will have gaps on either side of it. I think you meant that the rationals and irrationals are dense (for any two distinct numbers, you can find another number between them), which is also false for floating-point numbers.
As one who understands floats I really wish there was a better notation for literals, the best would be a floating literal in binary representation. For integers you can write 0x0B, 11, 0b1011 and have a very precise representation For floats you write 1e-1 or 0.1 and you get an ugly truncation. If it were possible to write something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)... people would be incentivate to use nice…
The initial precision doesn’t really matter, because if you plan to use this value in a computation, it will quickly accumulate an error, which you have to deal with anyway. There are three ways to deal with it: 1) ignore it, 2) account for it, 3) use numeric methods which retain it in a decent range. You may accidentally (1)==(3), but the problem doesn’t go away in general.
Earlier quoted context omitted.
The binary form is important to understand the implementation details. You even mention underflow. It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately. The binary form handily demonstrates the limitations. Understanding the floating point instructions is kinda optional but still valuable. Otherwise everyo…
It also explains why 0.1+0.2 is not 0.3. With binary IEEE-754 floats, none of those can be represented exactly[a]. With decimal IEEE-754 floats, it's possible, but the majority of hardware people interact with works on binary floats. [a]: Sure, if you `console.log(0.1)`, you'll get 0.1, but it's not possible to express it in binary exactly ; only after rounding. 0.5, however, is exactly representable.
Python 3.9.5
>>> 0.1.hex()
'0x1.999999999999ap-4'
>>> 0.2.hex()
'0x1.999999999999ap-3'
>>> (0.1 + 0.2).hex()
'0x1.3333333333334p-2'
>>> 0.3.hex()
'0x1.3333333333333p-2'Why is everyone complaining about people finding floats hard? Sure, scientific notation is easy to grasp, but you can't honestly tell me that it's trivial AFTER you consider rounding modes, subnormals, etc. Maybe if hardware had infinite precision like the real numbers nobody would be complaining ;) One thing I dislike in discussions about floats is this incessant focus on the binary representation. The representatio…
And so far the explanation in the article is the best I found, not least because subnormal numbers appear naturally.
There is a mathematical foundation behind it of course, but it is not easy for a programmer like me. I think it is better to think in term of bits and the integers they make, because that's what the computer sees. And going this way, you get NaN-boxing and serialization as a bonus.
Now, I tend to be most comfortable with a "machine first", bottom-up, low level approach to problems. Mathematical and architectural concepts are fine and all, but unless I have some idea about how it looks like in memory and the kind of instructions being run, I tend to feel lost. Some people may be more comfortable with high level reasoning, we don't all have the same approach, that's what I call real diversity and it is a good thing.
I don't understand how M * 2^E (modulo small representation details) is difficult to grasp. Then you have decimal types as M * 10^E. It's certainly much clearer than the windowing and bucketing in this article.
His approach works well for me. I don't retain arbitrary facts. At least part of this is a fear that if I don't properly understand its dynamics I will misapply it, better to discard it. The windowing explanation shows me a path from the intent of the designer through to the implementation (the algorithm). Now I can retain that knowledge.
At least part of this is a fear that if I don't properly understand its dynamics I will misapply it
The explanation that you like can’t save from it either. It’s not something you think through and just write the correct code. Quick exercise with the “new” knowledge you’ve got: you have an array of a million floats, add them up correctly:
Earlier quoted context omitted.
The binary form is important to understand the implementation details. You even mention underflow. It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately. The binary form handily demonstrates the limitations. Understanding the floating point instructions is kinda optional but still valuable. Otherwise everyo…
> It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately. Because you don't have all the digits available just for the mantissa? That seems quite intuitive to me, even if you don't know about the corner cases of FP. This isn't one of them.
I guess -- the one thing that explicitly looking at the bits does bring to the table, is the understanding that (of course) the number of bits or digits in the mantissa must be less than the number of bits or digits in an equivalent length integer. Of course, this is pretty obvious if you think about the fact that they are the same size in memory, but if we're talking about sizes in memory, then I guess we're talking about bits implicitly, so may as well make it explicit.
* actually, much more than just getting a general flavor, most of the time in numerical linear algebra stuff the actual bitwise representation is usually irrelevant, so you can get pretty far without thinking about bits.