Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's OK to compare floating-points for equality

lisyarus.github.io

Post reply on HN
Print view View original

111–120 of 150 posts

Re: It's OK to compare floating-points for equality

#111

Post by purplesyringa »

Negitivefrags wrote: post #108
dnautics wrote: post #62

> In reality it is a pretty deterministic (modulo compiler options, CPU flags, etc) IIRC this was not ALWAYS the case, on x86 not too long ago the CPU might choose to put your operation in an 80-bit fp register, and if due to multitasking the CPU state got evicted, it would only be able to store it in a 32-bit slot while it's waiting to be scheduled back in? It might not be the case now in a modern system if based on…

This is the kind of misinformation that makes people more wary of floats than they should be. The same series of operations with the same input will always produce exactly the same floating point results. Every time. No exceptions. Hardware doesn't matter. Breed of CPU doesn't matter. Threads don't matter. Scheduling doesn't matter. IEEE floating point is a standard. Everyone follows the standard. Anything not produc…

There's just so many "but"s to this that I can't in good faith recommend people to treat floats as deterministic, even though I'd very much love to do so (and I make such assumptions myself, caveat emptor):

- NaN bits are non-deterministic. x86 and ARM generate different sign bits for NaNs. Wasm says NaN payloads are completely unpredictable.

- GPUs don't give a shit about IEEE-754 and apply optimizations raging from DAZ to -ffast-math.

- sin, rsqrt, etc. behave differently when implemented by different libraries. If you're linking libm for sin, you can get different implementations depending on the libc in use. Or you can get different results on different hardware.

- C compilers are allowed to "optimize" a * b + c to FMA when they wish to. The standard only technically allows this merge within one expression, but GCC enables this in all cases by default on some `-std`s.

You're technically correct that floats can be used right, but it's just impossible to explain to a layman that, yes, floats are fine on CPUs, but not on GPUs; fine if you're doing normal arithmetic and sqrt, but not sin or rsqrt; fine on modern compilers, but not old ones; fine on x86, but not i686; fine if you're writing code yourself, but not if you're relying on linear algebra libraries, unless of course you write `a * b + c` and compile with the wrong options; fine if you rely on float equality, but not bitwise equality; etc. Everything is broken and the entire thing is a mess.

Re: It's OK to compare floating-points for equality

#114

Post by firebot »

jph wrote: post #7

I have this floating-point problem at scale and will donate $100 to the author, or to anyone here, who can improve my code the most. The Rust code in the assert_f64_eq macro is: if (a >= b && a - b I'm the author of the Rust assertables crate. It provides floating-point assert macros much as described in the article. https://github.com/SixArm/assertables-rust-crate/blob/main/s... If there's a way to make it more prec…

You probably don't need the (in)accuracy.

Fix your precision so it matches.

You only need so many significant digits.

Re: It's OK to compare floating-points for equality

#115

Post by Negitivefrags »

purplesyringa wrote: post #111
Negitivefrags wrote: post #108

Earlier quoted context omitted.

This is the kind of misinformation that makes people more wary of floats than they should be. The same series of operations with the same input will always produce exactly the same floating point results. Every time. No exceptions. Hardware doesn't matter. Breed of CPU doesn't matter. Threads don't matter. Scheduling doesn't matter. IEEE floating point is a standard. Everyone follows the standard. Anything not produc…

There's just so many "but"s to this that I can't in good faith recommend people to treat floats as deterministic, even though I'd very much love to do so (and I make such assumptions myself, caveat emptor): - NaN bits are non-deterministic. x86 and ARM generate different sign bits for NaNs. Wasm says NaN payloads are completely unpredictable. - GPUs don't give a shit about IEEE-754 and apply optimizations raging from…

Yes, there are a large number of ways to fall into traps that cause you to do a different series of operations when you didn't realise that you did. But that's still ultimately what all your examples are. (Except the NaN thing!)

I still think it's important to fight the misinformation.

Programmers have been conditioned to be so afraid of floats that many believe that doing a + b has an essentially random outcome when it doesn't work that way at all. It leads people to spend a bunch of effort on things that they don't need to be doing.

Re: It's OK to compare floating-points for equality

#116

Post by Asooka »

My one small nitpick is that vector length is usually 2 instructions with SSE4:

    dpps xmm0, xmm0, 0x17 ; dot product of 3 lanes, write lane 0
    sqrtss xmm0, xmm0
    ret
And is considerably faster than the fancy version, mainly because Intel still hasn't given us horizontal-max vector instruction! ARM is a bit better in that regard with their fancy vmaxvq_f32 and vmaxnmvq_f32...

Re: It's OK to compare floating-points for equality

#117

Post by Asooka »

amelius wrote: post #22

Think about this. It's silly to use floating point numbers to represent geometry, because it gives coordinates closer to the origin more precision and in most cases the origin is just an arbitrary point.

Well yeah, you would store your values in whatever representation fits your domain, then do the calculations with floats based on a suitable origin when needed. For example, for raytracing you would have each model defined in its local coordinate system with 32-bit floats for coordinates (because those are plenty accurate enough for single human-scale models), but offset them in the scene with 64-bit doubles (again, plenty enough of precision), and convert the ray coordinates to the local coordinates for ray-mesh intersection once the ray-box intersection passes.

Re: It's OK to compare floating-points for equality

#118

Post by [deleted] »

m-schuetz wrote: post #89
amelius wrote: post #22

Think about this. It's silly to use floating point numbers to represent geometry, because it gives coordinates closer to the origin more precision and in most cases the origin is just an arbitrary point.

For geometry, fixed-precision integers are better. But for computation and usability, floats are great. Scaling a 10 meter model in floats to 13% of the size is a trivial multiplication by 0.13f. With integers, this can get tricky. Can't first divide by 100 then multiply by 13 because you'd lose precision. Also can't multiply by 13 and then divide by 100 because you might overflow. Unless maybe venders would add hard…

[deleted]

Re: It's OK to compare floating-points for equality

#119

Post by MarkusQ »

vouwfietsman wrote: post #75
MarkusQ wrote: post #40

Earlier quoted context omitted.

> They don’t just lean into epsilons, the session context tolerance is used for almost every single point classification operation in geometric kernels and many primitives carry their own accumulating error component for downstream math. The GP wasn't wrong. To "lean in" means to fully commit to, go all in on, (or, equivalently, go all out on).

I think his point is: rather than "leaning into" it as in, masking through epsilons, he argues that tolerance is fundamental to the problem space, not a way to resolve edge cases.

Right. And my point is that "leaning in" doesn't mean masking, it means committing to. Taking seriously. Exactly the sort of thing he's describing.

I'm wondering if people have heard the expression "leaning in" from people who were insincere/lying, and assumed that that was what the phrase means?

Re: It's OK to compare floating-points for equality

#120

Post by _moof »

Something I've observed as someone who works in the physical sciences and used to work as a software engineer is:

Very few software engineers understand that tolerances are fundamental.

In the physical sciences, strict equality - of actual quantities, not variables in equations - is almost never a thing. Even though an equation might show that two theoretical quantities are exactly equal, the practical fact is that every quantity begins life as a measurement, and measurements have inherent uncertainty.

Even in design work, nothing is exact. It's simply not possible. A resistor's value is subject to manufacturing tolerances, will vary with temperature, and can drift as it ages. A mechanical part will also have manufacturing tolerance, and changes size and shape with temperature, applied forces, and wear. So even if a spec sheet states an exact number, the heading or notes will tell you that this is a nominal value under specific conditions. (Those conditions are also impossible to achieve and maintain exactly for all the same reasons.)

Even the voltages that represent 0 and 1 inside a computer aren't exact. Digital parts like CPUs, GPUs, RAM, etc. specify low and high thresholds, under or over which a voltage is considered a 0 or 1.

Floating-point numbers have uses outside the physical sciences, so there's no one-size-fits-all approach to using them correctly. But if you are writing code that deals with physical quantities, making equality comparisons is almost always going to be wrong even if floating-point numbers had infinite precision and no rounding error. Physical quantities simply can't be used that way.

Post reply on HN