Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Floating point visually explained (2017)

fabiensanglard.net

Post reply on HN
Print view View original

21–30 of 58 posts

Re: Floating point visually explained (2017)

#21

Post by xyzzy_plugh »

Sirenos wrote: post #19

Why is everyone complaining about people finding floats hard? Sure, scientific notation is easy to grasp, but you can't honestly tell me that it's trivial AFTER you consider rounding modes, subnormals, etc. Maybe if hardware had infinite precision like the real numbers nobody would be complaining ;) One thing I dislike in discussions about floats is this incessant focus on the binary representation. The representatio…

The binary form is important to understand the implementation details. You even mention underflow. It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately.

The binary form handily demonstrates the limitations. Understanding the floating point instructions is kinda optional but still valuable.

Otherwise everyone should just use varint-encoded arbitrary precision numbers.

Re: Floating point visually explained (2017)

#22

Post by sharikous »

As one who understands floats I really wish there was a better notation for literals, the best would be a floating literal in binary representation.

For integers you can write 0x0B, 11, 0b1011 and have a very precise representation

For floats you write 1e-1 or 0.1 and you get an ugly truncation. If it were possible to write something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)... people would be incentivate to use nice negative power of 2 floats, which are much less error prone than ugly base conversions.

This way you can overcame the fears around floats being nonexact and start writing more accurate tests (bit per bit) in many cases

Re: Floating point visually explained (2017)

#23

Post by mhh__ »

bullen wrote: post #13
adgjlsfhk1 wrote: post #10

Earlier quoted context omitted.

that's called fixed point. there isn't hardware for it because it is cheap to make in software using integer math.

But wouldn't it be sweet to have SIMD acceleration for fixed point? Or is that something you can do with integers and SIMD today? Say 4x4 matrix multiplication?

Are you asking if integer SIMD exists? (It does)

Re: Floating point visually explained (2017)

#24

Post by jacobolus »

sharikous wrote: post #22

As one who understands floats I really wish there was a better notation for literals, the best would be a floating literal in binary representation. For integers you can write 0x0B, 11, 0b1011 and have a very precise representation For floats you write 1e-1 or 0.1 and you get an ugly truncation. If it were possible to write something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)... people would be incentivate to use nice…

> better notation for literals [...] something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)

There are floating point hex literals. These can be written as 0x1p-1 == 0.5 and 0x1p-2 == 0.25.

You can use them in C/C++, Java, Julia, Swift, ..., but they are not supported everywhere.

https://observablehq.com/@jrus/hexfloat

Re: Floating point visually explained (2017)

#25

Post by colejohnson66 »

xyzzy_plugh wrote: post #21
Sirenos wrote: post #19

Why is everyone complaining about people finding floats hard? Sure, scientific notation is easy to grasp, but you can't honestly tell me that it's trivial AFTER you consider rounding modes, subnormals, etc. Maybe if hardware had infinite precision like the real numbers nobody would be complaining ;) One thing I dislike in discussions about floats is this incessant focus on the binary representation. The representatio…

The binary form is important to understand the implementation details. You even mention underflow. It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately. The binary form handily demonstrates the limitations. Understanding the floating point instructions is kinda optional but still valuable. Otherwise everyo…

It also explains why 0.1+0.2 is not 0.3. With binary IEEE-754 floats, none of those can be represented exactly[a]. With decimal IEEE-754 floats, it's possible, but the majority of hardware people interact with works on binary floats.

[a]: Sure, if you `console.log(0.1)`, you'll get 0.1, but it's not possible to express it in binary exactly; only after rounding. 0.5, however, is exactly representable.

Re: Floating point visually explained (2017)

#26

Post by stncls »

sharikous wrote: post #22

As one who understands floats I really wish there was a better notation for literals, the best would be a floating literal in binary representation. For integers you can write 0x0B, 11, 0b1011 and have a very precise representation For floats you write 1e-1 or 0.1 and you get an ugly truncation. If it were possible to write something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)... people would be incentivate to use nice…

I'm not sure this is what you are asking for, but would this be suitable?

> ISO C99 and ISO C++17 support floating-point numbers written not only in the usual decimal notation, such as 1.55e1, but also numbers such as 0x1.fp3 written in hexadecimal format. [...] The exponent is a decimal number that indicates the power of 2 by which the significant part is multiplied. Thus ‘0x1.f’ is 1 15/16, ‘p3’ multiplies it by 8, and the value of 0x1.fp3 is the same as 1.55e1.

https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html

Re: Floating point visually explained (2017)

#27

Post by jhgb »

xyzzy_plugh wrote: post #21
Sirenos wrote: post #19

Why is everyone complaining about people finding floats hard? Sure, scientific notation is easy to grasp, but you can't honestly tell me that it's trivial AFTER you consider rounding modes, subnormals, etc. Maybe if hardware had infinite precision like the real numbers nobody would be complaining ;) One thing I dislike in discussions about floats is this incessant focus on the binary representation. The representatio…

The binary form is important to understand the implementation details. You even mention underflow. It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately. The binary form handily demonstrates the limitations. Understanding the floating point instructions is kinda optional but still valuable. Otherwise everyo…

> It's difficult for most people to initially understand why you can't store a large number that can be represented by an equivalent size integer as a float accurately.

Because you don't have all the digits available just for the mantissa? That seems quite intuitive to me, even if you don't know about the corner cases of FP. This isn't one of them.

Re: Floating point visually explained (2017)

#28

Post by bspammer »

wodenokoto wrote: post #20

I was kinda hoping for a visualization of which numbers exists in floating point. While I always new about 0.1 + 0.2 -> 0.30000000000000004 it was still kind of an epiphany realizing that floating point numbers don’t so much have rounding errors as they are simply discreet numbers. You can move from one float to the next, which is a meaningfull operation on discreet numbers like integers, but not continuous numbers l…

I highly recommend watching this video, which explains floats in the context of Mario 64: https://www.youtube.com/watch?v=9hdFG2GcNuA

Games are a great way to explain floats because they're so visual. Specifically, check out 3:31, where the game has been hacked so that the possible float values in one of the movement directions is quite coarse.

(If you're curious what a PU is, and would like some completely useless knowledge, this video is also an absolute classic and very entertaining: https://www.youtube.com/watch?v=kpk2tdsPh0A)

Re: Floating point visually explained (2017)

#29

Post by brianzelip »

Wild, just got done listening to Coding Blocks podcast episode on data structure primitives where they go in depth on floating point, fixed point, binary floating point, and more. Great listen! See around 54min mark - https://www.codingblocks.net/podcast/data-structures-primiti...

Re: Floating point visually explained (2017)

#30

Post by adgjlsfhk1 »

jacobolus wrote: post #24
sharikous wrote: post #22

As one who understands floats I really wish there was a better notation for literals, the best would be a floating literal in binary representation. For integers you can write 0x0B, 11, 0b1011 and have a very precise representation For floats you write 1e-1 or 0.1 and you get an ugly truncation. If it were possible to write something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25)... people would be incentivate to use nice…

> better notation for literals [...] something like 1eb-1 (for 0.5) and 1eb-2 (for 0.25) There are floating point hex literals. These can be written as 0x1p-1 == 0.5 and 0x1p-2 == 0.25. You can use them in C/C++, Java, Julia, Swift, ..., but they are not supported everywhere. https://observablehq.com/@jrus/hexfloat

Julia is missing 32 bit and 16 bit hex floats unfortunately.
Post reply on HN