Yep, floats should be considered dangerous.
$ python3 -c 'print(.1 + .2)'
0.30000000000000004Floating point numbers, and why they suck
31–40 of 46 posts
Re: Floating point numbers, and why they suck
#32I'm very annoyed by this title. It's a beginners mistake and FP numbers just are what they are -- a tradeoff between precision, range and efficiency. If you use FP, you should be familiar with its intricacies in the same way a C programmer needs to be aware of unsigned overflows.
I agree. Enough already. Isn't this covered in something like week 2 of CS 101? If you've missed that, maybe the literally decades of "ermagerd, floats!" articles would give you a hint.
As someone who's worked in science and finance (modeling, not accounting), floats work just fine, thank you very much. The modeling/accounting split in finance is a legit point of confusion, though.
Re: Floating point numbers, and why they suck
#33Earlier quoted context omitted.
Decimal floating point still doesn’t have associative addition or multiplication.
But it can be shaped by rules to be associative enough to be useful for the common case. All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits (much like you'd guard against overflow on integer types). Once you need more range, you switch to a multi-word numeric format (or impose rounding rules).
> All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits
If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point.
The situations that make it useful to have a floating point (rather than fixed point) are also the situations where addition isn’t associative.
Re: Floating point numbers, and why they suck
#34Earlier quoted context omitted.
This. Sure, floating point numbers come with a footgun, but the title is so clickbaity, would it really hurt to name it: "What I wish I knew about floating numbers before relying on them?"
I think it's common to just not be taught floating points need to be handled a lot differently than whole numbers, so the common experience is getting bit by it. I don't remember ever being taught that in my traditional schooling, personally. This is a good link to send to people: https://floating-point-gui.de/
I remember covering how floating point were stored, but I don’t remember going all that deep into how that affected arithmetic.
Re: Floating point numbers, and why they suck
#35Earlier quoted context omitted.
But it can be shaped by rules to be associative enough to be useful for the common case. All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits (much like you'd guard against overflow on integer types). Once you need more range, you switch to a multi-word numeric format (or impose rounding rules).
> All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point. The situations that make it useful to have a floating point (rather than fixed point) are also the situations where addition isn’t associative.
> If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point.
You could, but why would you go to the extra complication (and ensuing bugs) of adding an implied fixed point to integers when a decimal float type gives it to you for free?
As I said, it's not a silver bullet; it's just for the majority case (much like 32 and 64-bit integers are for the majority case when dealing with whole numbers).
Re: Floating point numbers, and why they suck
#36Any vector addition done concurrently will result in indeterminate roundoff errors (to be precise there might be O(N!) different possible outputs depending on the order of calculation), this is literally the point of the field of numerical analysis. But that's really the tip of the iceberg. Beware any mathematical operation done with numbers of vastly different orders of magnitude. Beware spinning your own matmul even if you can't guarantee some normalcy to the numbers in your matrices.
Calculating an average is fairly painless, you can create an upper bound on the possible error that isn't usually dangerous. This isn't the case for all computations.
Re: Floating point numbers, and why they suck
#37The vast majority of problems with floating point numbers are simply due to people not understanding number bases. Only a tiny subset is caused by people not understanding floats proper.
Consider the number 1/3. In ternary (base 3), you write that as 0.1, whereas in decimal (base 10) you write it as 0.3333... recurring. If you try to represent that number with a fixed number of decimal places, you have precision issues. E.g. decimal 0.333 converts to 0.02222220210 in ternary.
Now, the thing with that example is that we treat decimal as a special, privileged representation, so we accept that 1/3 doesn't have a finite representation in decimal as an entirely natural fact of life, while we treat that same problem converting between decimal and binary as a fundamental deficiency of binary.
Let's talk about why some numbers have finite representations while others don't. 53/100 is written as 0.53 in decimal. The general rule is that if the denominator is a power of ten, you just write the numerator, then put the decimal mark have as many digits from the right as the power of ten. If the denominator is not a power of ten, you make it one. 1/2 turns into 5/10, which is 5 with one decimal place, or 0.5. Obviously, you can't actually do this for 1/3. There's no integer `a` where 3a is a power of ten. The general rule here is that, if the denominator has any prime factors not present in the base, you don't have a finite representation. 4/25 = 16/100 = 0.16 has a finite representation, but 5/7, 13/3 don't.
Now, because 3 is coprime with 10, no number with a finite ternary representation can have a finite decimal representation (and vice versa), and we're used to it. Where binary vs decimal becomes tricky and confuses people is that 10 = 2 * 5, so numbers that can be expressed with a power-of-two denominator have finite representations in both binary and decimal, so you can convert some numbers back and forth with no loss of precision. Numbers with a factor of 5 somewhere in their denominator can have finite decimal represntations (1/5 = 2/10 = 0.2), but can't have finite representations in binary. 0.1 = 1/10 = 1/(2 * 5) and you can't get rid of that five. And that's why everybody gets bitten by 0.1 seeming to be broken.
Re: Floating point numbers, and why they suck
#38I'm very annoyed by this title. It's a beginners mistake and FP numbers just are what they are -- a tradeoff between precision, range and efficiency. If you use FP, you should be familiar with its intricacies in the same way a C programmer needs to be aware of unsigned overflows.
IEEE 754 floating-point format has been very carefully and deliberately designed by the top experts in the field. When someone "it sucks", that is a guarantee that whatever design he might come up with would suck ten times as much.
Floating point feels to me like a very well designed jack-of-all-trades. They are decent for most tasks, but if you know exactly what you want to do then you can find something that does that particular job better.
Re: Floating point numbers, and why they suck
#39The problem is that we're still stuck with only binary floating point types in our CPUs and compilers and runtime environments, over a decade after ieee754 released their decimal float spec [1]. Once we finally move away from binary floats, these nasty binary-exponent-decimal-exponent lossy conversions will end (as will the horribly complicated nasty scanf and printf algorithms). This is a solved problem. Our actual…
I predict IPv4 will be widely considered legacy before decimal floating point numbers are common place, let alone binary floats being considered legacy.
From what I've seen, there's just simply no demand for decimal floating point; there's basically one or two people who ask for it, and this carries up all the way through to C committee standardization. Far more people care about things like non-default rounding mode or exception support than care about decimal floating point, and that's an area of floating point that itself is pretty damn low on people's priority lists.
The advantage of decimal floating points is that it fixes the rounding error when converting to/from decimal strings... and that's it. It doesn't fix non-associativity, it doesn't fix gradual precision loss, it doesn't fix portability issues because different libraries have different precision targets. And you get this for the slight cost of making every single floating point operation slower.
And here's the other truism about floating-point: most people who care about it value speed over correctness. There's strong demand for operations like approximate division that don't do the refinement steps, and consequently throw away half the bits. Fast-math flags are similarly pretty common, because users already aren't expecting bit-precise results, so the resulting changes in precision are quite often acceptable for them.
Re: Floating point numbers, and why they suck
#40I'm very annoyed by this title. It's a beginners mistake and FP numbers just are what they are -- a tradeoff between precision, range and efficiency. If you use FP, you should be familiar with its intricacies in the same way a C programmer needs to be aware of unsigned overflows.
The problem is that the creeping imprecision actually comes from the exponential mismatch, not the actual float itself. If we were inputting floats in binary format, most of our problems would disappear. But since we think in decimal, we input in decimal, and then magical complicated algorithms convert those "decimal" values to the "nearest" binary representation. And then the trouble ensues when we try to use them i…
> Decimal floats would operate just as we're used to, and would solve 95% of our floating point problems.
It does not in fact solve the problem identified in this blog post (a non-associative issue). It does not solve any of the problems I generally see mentioned in topics like multiplayer video game desyncs (math libraries on different platforms don't return the same results). It's a pretty bold assertion that "95% of our floating point problems" would be solved.