Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

0.1 and 0.2 Returns 0.30000000000000004 (2018)

qntm.org

Post reply on HN
Print view View original

131–140 of 161 posts

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#131

Post by dataflow »

crazygringo wrote: post #129
dataflow wrote: post #111

Earlier quoted context omitted.

There was a time when I thought (like you) that everybody should be using interval arithmetic, but then I came across a counterexample that convinced me I was wrong. I don't remember the precise example, but maybe the following will do the same for you. Say x = 4.0 ± 1.0. What is x / x? It should be x / x = 1.0 ± 0.0, but interval arithmetic will give you [3/5, 5/3]. Notice the interval is objectively wrong , as the…

Well, if it's built into the language and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem. Or if the library works on objects passed by reference and can confirm them as the same object, it could do that too. But if, in your code, you've copied x to y (not by reference), then it seems that x / y would correctly be [3/5, 5/3]. This is…

> and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem

You're moving the goalposts here. Those are HUGE if's. You went from something you could trivially implement in any language to something that requires a LOT of infrastructure and will severely limit your options.

How are you going to handle (x - z) / (x + z) when z = 0? How are you going to handle f(x) / g(x) when they turn out to compute the same value in different ways?

You go from "I need to change float to interval, give me 15 minutes" to "I need a computer algebra system, let me figure out if I can embed Mathematica/SymPy/Maple/etc. into program so I can do math." And even when you do that you STILL won't be able to handle cases where the symbolic engine can't simplify it for you. Which in general it won't be able to do.

> But if, in your code, you've copied x to y (not by reference), then it seems that x / y would correctly be [3/5, 5/3]. This is a feature, not a bug.

No, that is most definitely a bug. The correct result is 1.0, but you're producing [3/5, 5/3]. If that's not a bug to you then you might as well just output (-∞, +∞) everywhere and call it a day. You can insist on calling it a "feature" if it makes you feel better, but that won't change the fact that it's still just as useless (if not actively harmful) for your intended calculation as it was before.

Contrast this with just leaving it as a float instead of an interval, where you would've gotten the correct answer.

> In any case, since intervals are more likely to be more like ± 0.000000000000001, it doesn't seem like a problem in practice even if the compiler/interpreter doesn't optimize it away?

That's only after 1 iteration. Notice that in my example the error was multiplicative, not additive. Run more iterations and your error will magnify.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#132

Post by crazygringo »

dataflow wrote: post #131
crazygringo wrote: post #129

Earlier quoted context omitted.

Well, if it's built into the language and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem. Or if the library works on objects passed by reference and can confirm them as the same object, it could do that too. But if, in your code, you've copied x to y (not by reference), then it seems that x / y would correctly be [3/5, 5/3]. This is…

> and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem You're moving the goalposts here. Those are HUGE if's. You went from something you could trivially implement in any language to something that requires a LOT of infrastructure and will severely limit your options. How are you going to handle (x - z) / (x + z) when z = 0? How are yo…

I wasn't moving the goalposts, I was simply responding to your specific example.

But there's nothing "wrong" about it, it's correct -- that's how it's designed to work. If you're starting with non-extreme values and just miniscule floating-point errors and iterate 1,000 times with basic arithmetic I still don't see how it's going to cause a problem. E.g.:

  Math.pow((1 + Number.EPSILON), 1000) => 1.000000000000222
The result is the same even if you multiply in a loop 1,000 times rather than call Math.pow().

If you're multiplying a million or billion times then that's where I can now understand you should be an expert an numerical analysis in the first place to have any confidence in your result, and you know whether or not your float result can be trusted or not at all.

But the good thing is that if you don't know what you're doing, started with interval arithmetic and it ballooned to (-∞, +∞), then that's a strong signal you shouldn't necessarily be trusting the algorithm's results at all, and to go talk to someone with a background in numerical analysis, right?

Whereas if you iterate a million times and the interval is still miniscule compared to your values, you have absolute confidence you're fine. Seems useful to me -- not useless or actively harmful at all.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#133

Post by 29athrowaway »

crazygringo wrote: post #130
29athrowaway wrote: post #84

Earlier quoted context omitted.

You cannot compare floating point numbers like that. The equality test in floating point numbers is comparing against the epsilon. Math.abs(0.3 - (0.1 + 0.2)) Which is the same you other languages. Using the epsilon for comparison is not mentioned in the article. Floating point absorption is also not mentioned in the article. This entire discussion and the fact this is on the front page of HN is pretty disappointing…

Well, that was trivially easy to disprove with a little jiggling of numbers in the console: Math.abs(1.8 - (0.1 + 0.2 + 0.9 + 0.6)) returns false. Also, you generally really shouldn't be implementing any currency logic using floating point numbers, yikes. Stick to integers that represent the value in cents, or tenths of cents, or similar. Or, even better, a DECIMAL data type if your platform supports it. I genuinely…

I never said you should use floating-point numbers for currency. I said that if you are not aware of the shortcomings of floating-point numbers (the only numeric type in JavaScript, not counting workarounds like typed arrays) you should not be working with values representing currencies until you do.

Fixed-point numbers (aka "decimal" type in Java, C# and others) are the preferred way to deal with this problem as I mentioned in this same discussion hours ago.

Now, in the example you presented, you have a round-off error amplication problem where the round-off error grows larger than the epsilon. You can avoid that using the Kahan summation algorithm.

https://en.wikipedia.org/wiki/Kahan_summation_algorithm

    const arr = [0.1, 0.2, 0.9, 0.6];
    let sum = 0.0;
    let c = 0.0;
    for(let i = 0, l = arr.length; i
Now you evaluate sum, and it's 1.8 as expected.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#134

Post by dataflow »

crazygringo wrote: post #132
dataflow wrote: post #131

Earlier quoted context omitted.

> and the compiler or interpreter knows both x's are the same variable it could optimize it away to 1.0 ± 0.0, so I don't see the problem You're moving the goalposts here. Those are HUGE if's. You went from something you could trivially implement in any language to something that requires a LOT of infrastructure and will severely limit your options. How are you going to handle (x - z) / (x + z) when z = 0? How are yo…

I wasn't moving the goalposts, I was simply responding to your specific example. But there's nothing "wrong" about it, it's correct -- that's how it's designed to work. If you're starting with non-extreme values and just miniscule floating-point errors and iterate 1,000 times with basic arithmetic I still don't see how it's going to cause a problem. E.g.: Math.pow((1 + Number.EPSILON), 1000) => 1.000000000000222 The…

No, but I'm out of ideas for how to explain it any more clearly. If you really think this is all correct behavior and IA is going to make your life better, start using it in production and you'll learn it the hard way.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#135

Post by crazygringo »

dataflow wrote: post #134
crazygringo wrote: post #132

Earlier quoted context omitted.

I wasn't moving the goalposts, I was simply responding to your specific example. But there's nothing "wrong" about it, it's correct -- that's how it's designed to work. If you're starting with non-extreme values and just miniscule floating-point errors and iterate 1,000 times with basic arithmetic I still don't see how it's going to cause a problem. E.g.: Math.pow((1 + Number.EPSILON), 1000) => 1.000000000000222 The…

No, but I'm out of ideas for how to explain it any more clearly. If you really think this is all correct behavior and IA is going to make your life better, start using it in production and you'll learn it the hard way.

I guess I just don't see why it's worse than FP. I mean, you said:

> Contrast this with just leaving it as a float instead of an interval, where you would've gotten the correct answer.

But if I type into my console:

  var a = 0.1; var b = a + 0.2; b -= 0.2; b == a;
I get false. That's not correct. Whereas if "==" checked for overlap between intervals, it would be true. Which would be correct.

Heck, to use your own division example:

  (0.1 + 0.2 - 0.2) / 0.1 ==> 1.0000000000000002
That's not 1.0. So the FP you claim is so correct... just isn't at all. Again, while interval arithmetic would correctly detect overlap of intervals between that and 1.0.

Anyways, thanks for trying to explain.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#136

Post by Stupulous »

crazygringo wrote: post #135
dataflow wrote: post #134

Earlier quoted context omitted.

No, but I'm out of ideas for how to explain it any more clearly. If you really think this is all correct behavior and IA is going to make your life better, start using it in production and you'll learn it the hard way.

I guess I just don't see why it's worse than FP. I mean, you said: > Contrast this with just leaving it as a float instead of an interval, where you would've gotten the correct answer. But if I type into my console: var a = 0.1; var b = a + 0.2; b -= 0.2; b == a; I get false. That's not correct. Whereas if "==" checked for overlap between intervals, it would be true. Which would be correct. Heck, to use your own divi…

I think you run into the opposite issue, where equality is too likely and nonequal values return as equal. There might be a sweet spot with something like

(0.1 + 0.2 - 0.2) / 0.1 ==[+/- .001] 1.0 => True

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#137

Post by nomel »

kungito wrote: post #79

HN is more and more like first semester coding class where the professor always tells the "fun facts" but we have to be in the same class every year

You might be surprised how many people don’t understand the bit level basics these days. They’re not really the focus anymore, and they probably shouldn’t be. The point of advancing technology is to push the mundane, low level, difficulties away to make bigger concepts/abstractions easier to piece together and mentally bear.

From what I’ve seen with most recent grads, the education is shifting more and more towards algorithms, with experience mostly involving the use of existing libraries/frameworks, rather than lower level implementations that us “old timers” were forced to implement ourselves, thanks to lack of accessibility to freely usable code. I think GitHub, StackOverflow, and Google have changed the mental model of software development, significantly. I don’t think that’s a bad thing at all since it should free up some beans, especially for someone new to the field.

Not knowing this will bite you eventually, but it’s fairly trivial to work out.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#139

Post by enriquto »

dataflow wrote: post #134
crazygringo wrote: post #132

Earlier quoted context omitted.

I wasn't moving the goalposts, I was simply responding to your specific example. But there's nothing "wrong" about it, it's correct -- that's how it's designed to work. If you're starting with non-extreme values and just miniscule floating-point errors and iterate 1,000 times with basic arithmetic I still don't see how it's going to cause a problem. E.g.: Math.pow((1 + Number.EPSILON), 1000) => 1.000000000000222 The…

No, but I'm out of ideas for how to explain it any more clearly. If you really think this is all correct behavior and IA is going to make your life better, start using it in production and you'll learn it the hard way.

> If you really think this is all correct behavior and IA is going to make your life better, start using it in production and you'll learn it the hard way.

Even if this may come out as a bit snarky, this is a very good, honest, and helpful answer. At least, it would be helpful to extremely skeptical people like me. I'm a bit like crazygringo, and I cannot be convinced of anything until I have tried it myself and dirtied my hands with it.

To crazygringo: for a concrete, real-world example, try to implement a Kalman filter (a very simple numerical algorithm) using interval arithmetic. It simply doesn't work. You'll see that you cannot extract any useful result from the output of the computation. The implementation of an "interval Kalman filtering" is a (niche) subject of current research, where various very complicated algorithms are being invented to try to reproduce the nice properties of Kalman filtering--even when using something obviously inappropriate like interval arithmetic. For an intuitive understanding, assume that the input data are quantized to integer values, so that the starting intervals are of length 1.

Re: 0.1 and 0.2 Returns 0.30000000000000004 (2018)

#140

Post by worik »

caf wrote: post #92
chrisseaton wrote: post #64

Earlier quoted context omitted.

> Computers are finite state, floating point numbers are not. No, floating point numbers are finite state. That’s the whole point behind this discussion. There are only so many possible floating point numbers representable in so many bits. I never understand this confusion - you have finite memory - with this you can only represent a finite set of real numbers. So of course all the real numbers can’t be mapped direct…

I understand the confusion. It occurs when people haven't fully grokked that floating point numbers generally use binary representation, and that the set of numbers that can be represented with a finite number of decimal digits is distinct from the set of numbers that can be represented with a finite number of binary digits . People generally know that they can't write down the decimal value of 1÷3 exactly - they jus…

Sigh

The representation means that 2.9999999999 = 3.00000000

I am not confused. You cannot use equality in floating point. Why is anybody surprised by that? That is my point

(I did say floating point before when I ment real. A bit confused!)

Post reply on HN