Floats Are Weird
a.exozy.me
Floats Are Weird
1–10 of 120 posts
Re: Floats Are Weird
#2I’ve had weird float bugs that I didn’t have time for and fixed by operating on them as strings. Nowadays I’ve found pretty good libraries in common languages designed to handle the weird edges.
Re: Floats Are Weird
#3Nothing weird about it. It should be obvious that subtracting two floats that are very close to each other results in a loss of numerical precision:
1.000000003456e0 - 1.000000002345e0 = 0.000000001111e0 = 1.111numericalnoise e-9
It's exactly the same issue here. `math.exp(1e-15)` is `1.000000000000001`. If you subtract 1, you get 1 significant digit and numerical noise.
Re: Floats Are Weird
#4I guess "floats are weird" is a catchier title than "numerical computing is an acquired skill based in part on understanding the various consequences of value representation density being inversely proportional to the absolute value".
Re: Floats Are Weird
#5Then what's the best way to handle these cases? Are there any set of rules we should use while implementing the mathematical equations dealing with limits in floating points.
Re: Floats Are Weird
#6For this specific case, use Python's expm1, see https://docs.python.org/3/library/math.html#math.expm1 and history of expm1 at https://en.wikipedia.org/wiki/Exponential_function#expm1
def f(x):
return math.expm1(x)/x
>>> f(1e-15)
1.0000000000000007
1.0000000000000004
>>> math.nextafter(f(1e-15), 0)
1.0000000000000004
1.0000000000000005000000000000001666666666666667083333333333333416...Re: Floats Are Weird
#7Then what's the best way to handle these cases? Are there any set of rules we should use while implementing the mathematical equations dealing with limits in floating points.
For a summation, add the smaller numbers first. Smaller as in 0.0000000000053, not like -5172365126.
Re: Floats Are Weird
#8Nothing weird about it. It should be obvious that subtracting two floats that are very close to each other results in a loss of numerical precision: 1.000000003456e0 - 1.000000002345e0 = 0.000000001111e0 = 1.111numericalnoise e-9 It's exactly the same issue here. `math.exp(1e-15)` is `1.000000000000001`. If you subtract 1, you get 1 significant digit and numerical noise.
That's not what the post says if I understand correctly - the post explains why in certain situations the "noise" disappears, and in other cases it doesn't.
See comparison between f and g functions.
Re: Floats Are Weird
#9Then what's the best way to handle these cases? Are there any set of rules we should use while implementing the mathematical equations dealing with limits in floating points.
Floating point numbers have the highest precision when they're near 0. So if you know a number is going to be near 1, like exp(x) when x is small, then it's better to calculate the offset from 1 rather than the number itself. Programming languages make 'expm1' available for this purpose, which computes exp(x)-1.
Re: Floats Are Weird
#10Nothing weird about it. It should be obvious that subtracting two floats that are very close to each other results in a loss of numerical precision: 1.000000003456e0 - 1.000000002345e0 = 0.000000001111e0 = 1.111numericalnoise e-9 It's exactly the same issue here. `math.exp(1e-15)` is `1.000000000000001`. If you subtract 1, you get 1 significant digit and numerical noise.
The weird thing here is that the only change is making the denominator ln(exp(x)) instead of x. Catastrophic cancellation is still happening in the numerator (it’s still exp(x)-1), and the denominator winds up being some really tiny number.
It’s just that, due to the quirks of the floating point calculations involved, the numerator and denominator wind up being nearly the same noisy approximation to x, whereas in the original calculation that wasn’t true.