Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Floating point numbers, and why they suck

riskledger.com

Post reply on HN
Print view View original

41–46 of 46 posts

Re: Floating point numbers, and why they suck

#41

Post by adgjlsfhk1 »

okl wrote: post #26
longemen3000 wrote: post #17

In Julia, there is the `isapprox` function to inexactly compare 2 numbers. (you can use it in infix form: `x≈y`. By default, two numbers are approximately equal if their relative tolerance is less than `sprt(eps(typeof(x))` (around 1e-8 for 64bit Floating point numbers) Using equality to compare 2 floating point numbers doesn't get you anywhere, specially if you are using mathematical functions (the implementation of…

Relative tolerance becomes problematic though if your numbers are very close to zero. ( https://randomascii.wordpress.com/2012/02/25/comparing-float... )

Which is why it also lets you specify an absolute tolerance. That's not done by default because there isn't a good way to choose one (unlike with relative tolerance).

Re: Floating point numbers, and why they suck

#42

Post by kstenerud »

jcranmer wrote: post #40
kstenerud wrote: post #29

Earlier quoted context omitted.

The problem is that the creeping imprecision actually comes from the exponential mismatch, not the actual float itself. If we were inputting floats in binary format, most of our problems would disappear. But since we think in decimal, we input in decimal, and then magical complicated algorithms convert those "decimal" values to the "nearest" binary representation. And then the trouble ensues when we try to use them i…

> Decimal floats would operate just as we're used to, and would solve 95% of our floating point problems. It does not in fact solve the problem identified in this blog post (a non-associative issue). It does not solve any of the problems I generally see mentioned in topics like multiplayer video game desyncs (math libraries on different platforms don't return the same results). It's a pretty bold assertion that "95%…

Every single one of the problems in the article are due to binary-exponent / decimal-exponent mismatch and the inevitable differences in how different platforms do the conversions. With decimal floats, that problem disappears. There is no converted-it-to-90.34326374227855; all implementations would correctly have 90.34326 because there's no lossy conversion-to-binary-exponent to make; it's just stored in decimal with a decimal exponent, the end.

Multiply 90.34326 by 0.1 and then 0.1 again? Once again, all platforms will correctly and EXACTLY give the result 0.9034326. No loss of precision at all. Or do it 0.1 times 90.34326 times 0.1. Same result (exactly the same). Do that with binary floats and you're in for a world of hurt.

Re: Floating point numbers, and why they suck

#43

Post by jcranmer »

kstenerud wrote: post #42
jcranmer wrote: post #40

Earlier quoted context omitted.

> Decimal floats would operate just as we're used to, and would solve 95% of our floating point problems. It does not in fact solve the problem identified in this blog post (a non-associative issue). It does not solve any of the problems I generally see mentioned in topics like multiplayer video game desyncs (math libraries on different platforms don't return the same results). It's a pretty bold assertion that "95%…

Every single one of the problems in the article are due to binary-exponent / decimal-exponent mismatch and the inevitable differences in how different platforms do the conversions. With decimal floats, that problem disappears. There is no converted-it-to-90.34326374227855; all implementations would correctly have 90.34326 because there's no lossy conversion-to-binary-exponent to make; it's just stored in decimal with…

> Every single one of the problems in the article are due to binary-exponent / decimal-exponent mismatch and the inevitable differences in how different platforms do the conversions.

The article itself actually started with your assumption... and found it wasn't true. The type conversion wasn't the issue--it was coming back as the same value on all the different systems.

The flaw was that the sum came out wrong depending on the order that it was done. And the input numbers were (if I'm understanding correctly) computed as a / b, for some integer values a and b, which are not generally perfectly precise in decimal or binary floating point.

Re: Floating point numbers, and why they suck

#44

Post by kstenerud »

jcranmer wrote: post #43
kstenerud wrote: post #42

Earlier quoted context omitted.

Every single one of the problems in the article are due to binary-exponent / decimal-exponent mismatch and the inevitable differences in how different platforms do the conversions. With decimal floats, that problem disappears. There is no converted-it-to-90.34326374227855; all implementations would correctly have 90.34326 because there's no lossy conversion-to-binary-exponent to make; it's just stored in decimal with…

> Every single one of the problems in the article are due to binary-exponent / decimal-exponent mismatch and the inevitable differences in how different platforms do the conversions. The article itself actually started with your assumption... and found it wasn't true . The type conversion wasn't the issue--it was coming back as the same value on all the different systems. The flaw was that the sum came out wrong depe…

I think I need to do a blog post on this. I get the same kind of reaction from pretty much anyone I talk to about this, and it takes hours of explanations & diagrams to get them to understand. It's a real shame too, because a correct understanding would have a LOT more people putting pressure on implementors to support decimal floats :/

Re: Floating point numbers, and why they suck

#45

Post by longemen3000 »

okl wrote: post #26
longemen3000 wrote: post #17

In Julia, there is the `isapprox` function to inexactly compare 2 numbers. (you can use it in infix form: `x≈y`. By default, two numbers are approximately equal if their relative tolerance is less than `sprt(eps(typeof(x))` (around 1e-8 for 64bit Floating point numbers) Using equality to compare 2 floating point numbers doesn't get you anywhere, specially if you are using mathematical functions (the implementation of…

Relative tolerance becomes problematic though if your numbers are very close to zero. ( https://randomascii.wordpress.com/2012/02/25/comparing-float... )

yes, it is a common problem, it is acknowledged in the function's documentation: https://docs.julialang.org/en/v1/base/math/#Base.isapprox

Re: Floating point numbers, and why they suck

#46

Post by longemen3000 »

spicybright wrote: post #22
pjam wrote: post #19

Earlier quoted context omitted.

This. Sure, floating point numbers come with a footgun, but the title is so clickbaity, would it really hurt to name it: "What I wish I knew about floating numbers before relying on them?"

I think it's common to just not be taught floating points need to be handled a lot differently than whole numbers, so the common experience is getting bit by it. I don't remember ever being taught that in my traditional schooling, personally. This is a good link to send to people: https://floating-point-gui.de/

also: https://0.30000000000000004.com/
Post reply on HN