Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's OK to compare floating-points for equality

lisyarus.github.io

Post reply on HN
Print view View original

101–110 of 150 posts

Re: It's OK to compare floating-points for equality

#101

Post by russdill »

hmry wrote: post #17
jph wrote: post #7

I have this floating-point problem at scale and will donate $100 to the author, or to anyone here, who can improve my code the most. The Rust code in the assert_f64_eq macro is: if (a >= b && a - b I'm the author of the Rust assertables crate. It provides floating-point assert macros much as described in the article. https://github.com/SixArm/assertables-rust-crate/blob/main/s... If there's a way to make it more prec…

Is there any constant more misused in compsci than ieee epsilon? :) It's defined as the difference between 1.0 and the smallest number larger than 1.0. More usefully, it's the spacing between adjacent representable float numbers in the range 1.0 to 2.0. Because floats get less precise at every integer power of two, it's impossible for two numbers greater than or equal to 2.0 to be epsilon apart. The spacing between 2…

It would be very useful to be able to compare the significant directly then. I realize there is a boundary issue when a significant is very close to 0x00..000 or 0xFFF..FFF

Re: It's OK to compare floating-points for equality

#102

Post by Dylan16807 »

dnautics wrote: post #62

> In reality it is a pretty deterministic (modulo compiler options, CPU flags, etc) IIRC this was not ALWAYS the case, on x86 not too long ago the CPU might choose to put your operation in an 80-bit fp register, and if due to multitasking the CPU state got evicted, it would only be able to store it in a 32-bit slot while it's waiting to be scheduled back in? It might not be the case now in a modern system if based on…

> IIRC this was not ALWAYS the case, on x86 not too long ago the CPU might choose to put your operation in an 80-bit fp register, and if due to multitasking the CPU state got evicted, it would only be able to store it in a 32-bit slot while it's waiting to be scheduled back in?

I don't think the CPU was ever allowed to do that, but with your average compiler you were playing with fire.

Did any actual OS mess up state like that? They could and should save the full registers. There's even a bultin instruction for this, FSAVE.

Re: It's OK to compare floating-points for equality

#103

Post by kbolino »

ack_complete wrote: post #97
kbolino wrote: post #84

Earlier quoted context omitted.

There's another gotcha. Consider positive, normal x and y where ulp(y) != ulp(x). Bitwise comparison, regardless of tolerance, will consider x to be far from y, even though they might be adjacent numbers, e.g. if y = x+ulp(x) but y is a power of 2.

This case actually works because for finite numbers of a given sign, the integer bit representations are monotonic with the value due to the placement of the exponent and mantissa fields and the implicit mantissa bit. For instance, 1.0 in IEEE float is 0x3F800000, and the next immediate representable value below it 1.0-e is 0x3F7FFFFF. Signed zero and the sign-magnitude representation is more of an issue, but can be…

I interpreted OP's "bit-cast to integer, strip few least significant bits and then compare for equality" message as suggesting this kind of comparison (Go):

  func equiv(x, y float32, ignoreBits int) bool {
      mask := uint32(0xFFFFFFFF)
with the sensitivity controlled by ignoreBits, higher values being less sensitive.

Supposing y is 1.0 and x is the predecessor of 1.0, the smallest value of ignoreBits for which equiv would return true is 24.

But a worst case example is found at the very next power of 2, 2.0 (bitwise 0x40000000), whose predecessor is quite different (bitwise 0x3FFFFFFF). In this case, you'd have to set ignoreBits to 31, and thus equivalence here is no better than checking that the two numbers have the same sign.

Re: It's OK to compare floating-points for equality

#104

Post by alterom »

jstanley wrote: post #47
throwup238 wrote: post #34

Earlier quoted context omitted.

> This is why most geometry kernels (see open cascade) sport things like "fuzzy boolean operations" [0]) that lean into epsilons. These epsilons mask the error-prone supply chain of these meshes that arrive in your program by allowing some tolerance. They don’t just lean into epsilons, the session context tolerance is used for almost every single point classification operation in geometric kernels and many primitives…

> This is a fundamentally unsolvable problem with floating point math It's a fundamentally unsolvable problem with B-reps ! The problem completely disappears with F-reps. (In exchange for some other difficult problems).

>(In exchange for some other difficult problems).

Ahhaha.

(I used to work in nTop, and boy is this an understatement when it comes to field based solid modeling)

Re: It's OK to compare floating-points for equality

#105

Post by Dylan16807 »

amelius wrote: post #22

Think about this. It's silly to use floating point numbers to represent geometry, because it gives coordinates closer to the origin more precision and in most cases the origin is just an arbitrary point.

Floating point has the benefit of not screaming and exploding when you have to take three lengths and calculate a volume.

Double precision floating point is like a 54-bit fixed point system that automatically scales to the exact size you need it to be. You get huge benefits for paying those 10 exponent bits. Even if you need those extra bits, you're often better off switching to a higher precision float or a double-double system.

Re: It's OK to compare floating-points for equality

#106

Post by Dylan16807 »

adrian_b wrote: post #54
valicord wrote: post #50

Earlier quoted context omitted.

Wait this doesn't make sense. Yes you'd get smaller absolute error in radians, but it doesn't really help because it's different units. Relative error is the same in degrees and radians, that's the whole point of exponential representation. All you're doing is adding a fixed offset to the exponent, but it doesn't give you any more precision when converting to radians

Having a constant relative error is indeed the reason for using floating-point numbers. However, for angles the relative error is completely irrelevant. For angles only the absolute error matters. For angles the optimum representation is as fixed-point numbers, not as floating-point numbers.

The absolute error accounting for units is what matters.

Changing the unit gives the illusion of changing absolute error, but doesn't actually change the absolute error.

Re: It's OK to compare floating-points for equality

#107

Post by spacechild1 »

Cold_Miserable wrote: post #95

I've yet to encounter a need for == equality for floating point operations.

I need it all the time. A very common case is caching of (expensive) computations. Let's say you have a parameter for an audio plugin and every parameter change requires some non-trivial and possibly expensive computation (e.g. calculation of filter coefficients). To avoid wasting CPU cycles on every audio sample, you only do the (re)calculation when the parameter has actually changed:

  double freq = getInput(0);
  if (freq != mLastFreq) {
    calculateCoefficients(freq);
    mLastFreq = freq;
  }
Also, keep in mind that certain languages, such as JS, store all numbers as double-precision floating point numbers. So every time you are writing a numeric for-loop in JS you are implicitly relying on floating point equality :)

Re: It's OK to compare floating-points for equality

#108

Post by Negitivefrags »

dnautics wrote: post #62

> In reality it is a pretty deterministic (modulo compiler options, CPU flags, etc) IIRC this was not ALWAYS the case, on x86 not too long ago the CPU might choose to put your operation in an 80-bit fp register, and if due to multitasking the CPU state got evicted, it would only be able to store it in a 32-bit slot while it's waiting to be scheduled back in? It might not be the case now in a modern system if based on…

This is the kind of misinformation that makes people more wary of floats than they should be.

The same series of operations with the same input will always produce exactly the same floating point results. Every time. No exceptions.

Hardware doesn't matter. Breed of CPU doesn't matter. Threads don't matter. Scheduling doesn't matter. IEEE floating point is a standard. Everyone follows the standard. Anything not producing indentical results for the same series of operations is *broken*.

What you are referring to is the result of different compilers doing a different series of operations than each other. In particular, if you are using the x87 fp unit, MSVC will round 80-bit floating point down to 32/64 bits before doing a comparison, and GCC will not by default.

Compliers doesn't even use 80-bit FP by default when compiling for 64 bit targets, so this is not a concern anymore, and hasn't been for a very long time.

Re: It's OK to compare floating-points for equality

#109

Post by kbolino »

oasisaimlessly wrote: post #98
kbolino wrote: post #84

Earlier quoted context omitted.

There's another gotcha. Consider positive, normal x and y where ulp(y) != ulp(x). Bitwise comparison, regardless of tolerance, will consider x to be far from y, even though they might be adjacent numbers, e.g. if y = x+ulp(x) but y is a power of 2.

I don't think this is true. Modulo the sign bit, the "next float" operator is equivalent to the next bitstring or the integer++.

Sure, but that operator can propagate a carry all the way to the most significant bit, so a check for bitwise equality after "strip[ping] few least significant bits" will yield false in some cases. The pathologically worst case for single precision, for example, is illustrated by the value 2.0 (bitwise 0x40000000) and its predecessor, which differ in all bits except the sign.

Re: It's OK to compare floating-points for equality

#110

Post by Dylan16807 »

ryandrake wrote: post #38
rpdillon wrote: post #15

Earlier quoted context omitted.

Yeah, I'm not sure how widespread the knowledge is that floating point trades precision for magnitude. Its obvious if you know the implementation, but I'm not sure most folks do.

I remember having convincing a few coworkers that the number of distinct floating point values between 0.0 and 1.0 is the same as the number of values between 1.0 and infinity. They must not be teaching this properly anymore. Are there no longer courses that explain the basics of floating point representation? I was arguing that we could squeeze a tiny bit more precision out of our angle types by storing angles in ra…

That doesn't work. The only real difference between those two scales is in the values located between -.0000000001 and .0000000001 And that's grossly underestimating the number of 0s.

No matter what scale you pick, your number line is going to look like this: https://anniecherkaev.com/images/floating_point_density.jpg Do a 2x zoom in or out and not a single pixel of the graph will change, just the labels.

Whether your biggest value is 0.005 or 7000000, most of your range has 25 (or 54) bits of precision. 99% of values are either too small to matter or outside your range. Changing your scale shifts between the "too small" and "too big" categories, but the number of useful values stays roughly the same.

Post reply on HN