Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

0.1 and 0.2 Returns 0.30000000000000004 (2018)

qntm.org

Post reply on HN
Print view View original

121–130 of 161 posts

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#121

Post by lifthrasiir »

lmilcin wrote: post #99
lifthrasiir wrote: post #87

Earlier quoted context omitted.

> 99% applications, if you don't do anything stupid you are completely fine. This is a common misconception. Or "anything stupid" is quite broader than you think. The main issue with FP arithmetic is that effectively every calculation incurs an error. You seem to aware of catastrophic cancellation, but it is problematic not because of the cancellation but because the error is proportional to the input, which can wild…

I work on financial risk calculations during day and embedded control devices after my official work hours. My controller running moving horizon estimator to simulate thermal system of the espresso machine and Kalman filters to correct model parameters against measurements runs 50 times a second and works fine for days, thank you, using floats on STM32. I have written huge amount of embedded software over past 20 yea…

Maybe I should have directly mentioned that "some guard against FP errors" includes a generic guard against errors. If your system processes messy data then you probably want some guards anyway and FP errors are just one (relatively small) class of errors. In fact I would be more worried if embedded devices don't have such guards than if they do faulty FP calculations.

There are also many classes of softwares where FP inaccuracy is simply no problem---such as 3D rendering where the worst FP error is probably z-fighting. Not every application of FP requires "correct" calculation.

> What newer games do, they allow client and server run simulations independently even when they don't agree exactly, and then reconcile the results from time to time.

That's not what "newer" games do, it's what games that can withstand the trade-off do. In particular if the outcome of a particular action is crucial to players you can't reconcile that (projectiles are canonical examples); once the player observed the outcome it's done. Reconcilation thus typically happens in the animation level, so that it can hide the delay until the outcome is observed. This does not actually require the simulation---an simple interpolation is frequent AFAIK---so your point is irrelevant.

There are several genres of games where the lockstep simulation remains pretty much the only way, and thus FP is discouraged. Not because FP is expensive, which had been false for quite a long time anyway (no game developer uses the "fast" inverse square root algorithm any more).

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#122

Post by Dylan16807 »

generalizations wrote: post #113
dataflow wrote: post #111

Earlier quoted context omitted.

There was a time when I thought (like you) that everybody should be using interval arithmetic, but then I came across a counterexample that convinced me I was wrong. I don't remember the precise example, but maybe the following will do the same for you. Say x = 4.0 ± 1.0. What is x / x? It should be x / x = 1.0 ± 0.0, but interval arithmetic will give you [3/5, 5/3]. Notice the interval is objectively wrong , as the…

Your example only proves your point if every instance of x is the same x, with the same objective value. i.e., what if x/x is actually (x=5.0)/(x=3.0) ?

x can't be two values at the same time. It's one instance, not a class.

If you have "scope1.x/scope2.x" then they don't cancel out, but that's not "x/x"

And if you save a value for later, and change x, then that value isn't x anymore.

(I assume you're using = to state the value, not as an assignment operator.)

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#123

Post by FabHK »

AaronFriel wrote: post #34
crazygringo wrote: post #20

Indeed, and therefore: 0.1 + 0.2 != 0.3 You can check it in the JavaScript console. This actually makes me wonder if anyone's ever attempted a floating-point representation that builds in an error range , and correctly propagated/amplified error over operations. E.g. a simple operation like "1 / 10" (to generate 0.1) would be stored not as a single floating-point value, but really as the range between the closest rep…

Interval arithmetic is what you're looking for, and there's an IEEE standard and many implementations.

Another way to get a feeling for the error (simpler than fully fledged interval arithmetic) is to toggle the rounding rules from the usual (towards even, or so) to up and down, and observe the change in result.

https://en.wikipedia.org/wiki/IEEE_754#Rounding_rules

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#124

Post by PaulHoule »

AaronFriel wrote: post #34
crazygringo wrote: post #20

Indeed, and therefore: 0.1 + 0.2 != 0.3 You can check it in the JavaScript console. This actually makes me wonder if anyone's ever attempted a floating-point representation that builds in an error range , and correctly propagated/amplified error over operations. E.g. a simple operation like "1 / 10" (to generate 0.1) would be stored not as a single floating-point value, but really as the range between the closest rep…

Interval arithmetic is what you're looking for, and there's an IEEE standard and many implementations.

That is overthinking it horribly.

The problem is that the user wants to write 1/10 and 2/20 and 3/10 but those numbers aren't really in the binary system.

The user gets some numbers (let's call them A, B and C) that aren't the same but they fool people at first because they not only deserialize as 0.1 but the they also serialize from 0.1. Trouble is that A + B != C but some other number.

Excel tries to hide it but the real answer is to keep the exponent in base 10 if you plan to read and write numbers like 137.036 or 9.1E-31. How the mantissa is doesn't matter, it could be base 7 for all I care -- it is just an integer.

Interval math is for much tougher problems like recursion of

  k * x * (1-x)
is easily proven to have periodic orbits of infinitely long period, but if you are using 32-bit floats you can't have a period longer than 4 billion. That kind of qualitatively difference means that there's no scientific value in iterating that function with floats, although you can do accurate grid samples with interval arithmetic.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#125

Post by kazinator »

lmilcin wrote: post #46
kazinator wrote: post #16

This just has to do with printing. This is the TXR Lisp interactive listener of TXR 256. Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet. TXR works even if the application surface is not free of dirt and grease. 1> (+ 0.1 0.2) 0.3 OK, so then: 2> (set *print-flo-precision* 17) 17 3> (+ 0.1 0.2) 0.30000000000000004 But: 4> 0.1 0.10000000000000001 5> 0.2 0.20000000000000001 6> 0.3 0.2999999999999999…

> The misleading action is to compare the input notation of 0.1 and 0.2 to the printed output of the sum, rather than consistently compare nothing but values printed using the same precision. I think the problem is the act of caring for the least significant bits. If you care for least significant bits of a floating point number it means you are doing something wrong. FP numbers should be treated as approximations. M…

Problem is, most people think that 0.3 being an approximation refers to the fact that it's one significant figure measurement of some sort, like 0.3V on a multimeter. Not that it's inherently an approximation.

Pencil-and-paper floating-point numbers like 1.23 x 10^5 are approximations of measurements (if we are doing science or engineering), but are inherently exact. Calculators bear that out, because calculators use base 10 floating-point, like pencil-and-paper calculations.

0.3 being inexact is only an artifact of the floating-point system being in a different base. No matter how many digits we throw at it, we cannot represent 0.3 in binary floating point. Not 64 bits, not 1024 bits, not 65535 bits.

If we use binary notation for floating-point numbers, they likewise become exact, in terms of representation. The inexactness we deal with then is the familiar type that we know from pencil-and-paper calculations: truncation to a certain number of digits after performing an operation like addition or multiplication.

But that truncation will not happen in a calculation in which both input operands are exactly represented, and the result is also exactly representable!!!

If base ten were used, 0.1 + 0.2 would be 0.3, exactly.

If we use power-of-two values, and combinations thereof, we don't have the problem:

  1> (= 0.625 (+ 0.125 0.5))
  t
No problem.

  2> (set *print-flo-precision* 17)
  17
  3> 0.625
  0.625
  4> 0.5
  0.5
  5> 0.125
  0.125
  6> (+ 0.125 0.5)
  0.625
No junk digits.

  7> (= 0.25 (sqrt 0.0625))
  t
Wee ...

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#126

Post by neolog »

29athrowaway wrote: post #84
crazygringo wrote: post #20

Indeed, and therefore: 0.1 + 0.2 != 0.3 You can check it in the JavaScript console. This actually makes me wonder if anyone's ever attempted a floating-point representation that builds in an error range , and correctly propagated/amplified error over operations. E.g. a simple operation like "1 / 10" (to generate 0.1) would be stored not as a single floating-point value, but really as the range between the closest rep…

You cannot compare floating point numbers like that. The equality test in floating point numbers is comparing against the epsilon. Math.abs(0.3 - (0.1 + 0.2)) Which is the same you other languages. Using the epsilon for comparison is not mentioned in the article. Floating point absorption is also not mentioned in the article. This entire discussion and the fact this is on the front page of HN is pretty disappointing…

> have you ever implemented any logic involving currency? You may want to take another look at it.

Floating point arithmetic is good enough for science, should be good enough for commerce too, no? Why is commerce special?

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#128

Post by lamp987 »

29athrowaway wrote: post #84
crazygringo wrote: post #20

Indeed, and therefore: 0.1 + 0.2 != 0.3 You can check it in the JavaScript console. This actually makes me wonder if anyone's ever attempted a floating-point representation that builds in an error range , and correctly propagated/amplified error over operations. E.g. a simple operation like "1 / 10" (to generate 0.1) would be stored not as a single floating-point value, but really as the range between the closest rep…

You cannot compare floating point numbers like that. The equality test in floating point numbers is comparing against the epsilon. Math.abs(0.3 - (0.1 + 0.2)) Which is the same you other languages. Using the epsilon for comparison is not mentioned in the article. Floating point absorption is also not mentioned in the article. This entire discussion and the fact this is on the front page of HN is pretty disappointing…

rounding error after multiple operations can be more than just epsilon

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#129

Post by crazygringo »

dataflow wrote: post #111
crazygringo wrote: post #76

Earlier quoted context omitted.

But if the intervals are growing to infinity, then should you be trusting your result at all? Are there really cases where current FP arithmetic gives an accurate result, but where the error bounds of interval arithmetic would grow astronomically? It seems like you'd have to trust FP rounding to always cancel itself out in the long run instead of potentially accumulating more and more bias with each iteration. Is tha…

There was a time when I thought (like you) that everybody should be using interval arithmetic, but then I came across a counterexample that convinced me I was wrong. I don't remember the precise example, but maybe the following will do the same for you. Say x = 4.0 ± 1.0. What is x / x? It should be x / x = 1.0 ± 0.0, but interval arithmetic will give you [3/5, 5/3]. Notice the interval is objectively wrong , as the…

Well, if it's built into the language and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem. Or if the library works on objects passed by reference and can confirm them as the same object, it could do that too.

But if, in your code, you've copied x to y (not by reference), then it seems that x / y would correctly be [3/5, 5/3]. This is a feature, not a bug.

In any case, since intervals are more likely to be more like ± 0.000000000000001 when dealing with basic common non-iterative calculations, it doesn't seem like a problem in practice even if the compiler/interpreter doesn't optimize it away?

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#130

Post by crazygringo »

29athrowaway wrote: post #84
crazygringo wrote: post #20

Indeed, and therefore: 0.1 + 0.2 != 0.3 You can check it in the JavaScript console. This actually makes me wonder if anyone's ever attempted a floating-point representation that builds in an error range , and correctly propagated/amplified error over operations. E.g. a simple operation like "1 / 10" (to generate 0.1) would be stored not as a single floating-point value, but really as the range between the closest rep…

You cannot compare floating point numbers like that. The equality test in floating point numbers is comparing against the epsilon. Math.abs(0.3 - (0.1 + 0.2)) Which is the same you other languages. Using the epsilon for comparison is not mentioned in the article. Floating point absorption is also not mentioned in the article. This entire discussion and the fact this is on the front page of HN is pretty disappointing…

Well, that was trivially easy to disprove with a little jiggling of numbers in the console:

  Math.abs(1.8 - (0.1 + 0.2 + 0.9 + 0.6))
returns false.

Also, you generally really shouldn't be implementing any currency logic using floating point numbers, yikes. Stick to integers that represent the value in cents, or tenths of cents, or similar. Or, even better, a DECIMAL data type if your platform supports it.

I genuinely hope you've never written financial software that judges if the results of two calculations are equal via the method you've described.

Post reply on HN