Since we're on the subject of floating-point numbers in Lisp/Scheme: https://wukix.com/lisp-decimals
This uses the Lisp ratio type in place of floats, to avoid the usual float issues.
ntoronto, any thoughts?
Practically Accurate Floating-Point Math
21–30 of 33 posts
Re: Practically Accurate Floating-Point Math
#22Since we're on the subject of floating-point numbers in Lisp/Scheme: https://wukix.com/lisp-decimals This uses the Lisp ratio type in place of floats, to avoid the usual float issues. ntoronto, any thoughts?
ratio's are good to have in any language, or as a library, but surely it's slower, and more memory is wasted (also hard to argue about how much exactly), and it still has to settle at some approximations for irrational numbers like PI which is used quite a lot when floating point numbers are (or at least in my experience).
Re: Practically Accurate Floating-Point Math
#23Since we're on the subject of floating-point numbers in Lisp/Scheme: https://wukix.com/lisp-decimals This uses the Lisp ratio type in place of floats, to avoid the usual float issues. ntoronto, any thoughts?
ratio's are good to have in any language, or as a library, but surely it's slower, and more memory is wasted (also hard to argue about how much exactly), and it still has to settle at some approximations for irrational numbers like PI which is used quite a lot when floating point numbers are (or at least in my experience).
All correct, for exact rationals. Racket's exact rational arithmetic tends to take about 1000x the amount of time floating-point arithmetic takes to compute similar functions, and creates a lot of garbage on the heap. It gets worse, though: with long-running computations, exact rationals' numerators and denominators tend to grow without bound, unless you explicitly round them to a fixed precision. If you take the latter path, you might as well use bigfloats, which wrap MPFR's multi-precision floats, and are faster.
Just looking at the Wu-Decimal page, though, I can't tell whether they're internally exact rationals or base-10 floats. If they're base-10 floats, they have all the same issues base-2 floats have. If they're internally exact rationals, I wonder what they do for division, which the set D isn't closed under.
Re: Practically Accurate Floating-Point Math
#24If you are interested in writing accurate numerical algorithms, as further reading I highly recommend reading "Accuracy and Stability of Numerical Algorithms" by Nick Higham, which is a very good comprehensive book. Like this article says, floating-point arithmetic is often thought of as mysterious, but it really isn't: it just obeys its own precisely specified rules. I think it's very good to dispel the mystery, so…
The first thing you should worry about is whether your computation is ill-conditioned, and only then worry whether it's numerically stable.
OK, but how does one apply this advice in practice?
The radiative exchange between two surfaces goes as x^4-y^4 where x and y are the surface temperatures. I can worry about whether that's ill-conditioned all I want, but at the end of the day I'm further ahead evaluating (x-y)(x+y)(x^2+y^2) than the original expression.
Similarly, if I need to find x^2-1, what recourse do I have? Maybe there's some reduction that makes the computation more stable. But normally I would expect to discover such a reduction by turning x^2-1 into (x-1)(x+1), and then seeing if one of those factors cancels out somewhere. So it still seems like a good first step.
Re: Practically Accurate Floating-Point Math
#25Earlier quoted context omitted.
ratio's are good to have in any language, or as a library, but surely it's slower, and more memory is wasted (also hard to argue about how much exactly), and it still has to settle at some approximations for irrational numbers like PI which is used quite a lot when floating point numbers are (or at least in my experience).
All correct, for exact rationals. Racket's exact rational arithmetic tends to take about 1000x the amount of time floating-point arithmetic takes to compute similar functions, and creates a lot of garbage on the heap. It gets worse, though: with long-running computations, exact rationals' numerators and denominators tend to grow without bound, unless you explicitly round them to a fixed precision. If you take the lat…
Wu-Decimal uses exact rationals (they aren't base 10 floats). Division works according to normal Lisp semantics, since the CL ratio type is used for arithmetic. Let's say you divide something by 3 and now you have infinitely repeating digits: then it is no longer in set D, and Wu-Decimal no longer considers it to be of decimal type. Instead, it is treated as a fraction, again per standard CL semantics. The "Printing" example tries to clarify this (notice that 1/2 prints as '0.5' but 1/3 remains '1/3').
Re: Practically Accurate Floating-Point Math
#26Earlier quoted context omitted.
ratio's are good to have in any language, or as a library, but surely it's slower, and more memory is wasted (also hard to argue about how much exactly), and it still has to settle at some approximations for irrational numbers like PI which is used quite a lot when floating point numbers are (or at least in my experience).
All correct, for exact rationals. Racket's exact rational arithmetic tends to take about 1000x the amount of time floating-point arithmetic takes to compute similar functions, and creates a lot of garbage on the heap. It gets worse, though: with long-running computations, exact rationals' numerators and denominators tend to grow without bound, unless you explicitly round them to a fixed precision. If you take the lat…
If they're base-10 floats, they have all the same issues base-2 floats have
Nitpick: although base 10 floats have similar issues related to being of limited precision, they are superior to base 2 floats in the aspect of representing decimal numbers without the binary approximation: e.g. 1/10 is nonrepeating as 1.0 x 10^-1 but infinitely repeating as binary, so for IEEE 754 binary32 you get 1.10011001100110011001101 (1.60000002384185791015625) x 2^-4.
Re: Practically Accurate Floating-Point Math
#27Earlier quoted context omitted.
All correct, for exact rationals. Racket's exact rational arithmetic tends to take about 1000x the amount of time floating-point arithmetic takes to compute similar functions, and creates a lot of garbage on the heap. It gets worse, though: with long-running computations, exact rationals' numerators and denominators tend to grow without bound, unless you explicitly round them to a fixed precision. If you take the lat…
If they're base-10 floats, they have all the same issues base-2 floats have Nitpick: although base 10 floats have similar issues related to being of limited precision, they are superior to base 2 floats in the aspect of representing decimal numbers without the binary approximation: e.g. 1/10 is nonrepeating as 1.0 x 10^-1 but infinitely repeating as binary, so for IEEE 754 binary32 you get 1.10011001100110011001101 (…
Yes, but they're about as bad for representing base 12 numbers, so it evens out.
Re: Practically Accurate Floating-Point Math
#28Sorry to tell you, but 99% of so called expert developers don't understand this.
For them a real number is a a float (exactly). And I have been fired for saying e-commerce should be done with fixed point numbers. (we do + - * / at most)
Headaches really comes with multi currency web sites since conversion brings incommesurability.
They also told me that since we are using linear equation most of the time (are we?), errors should dissipates like the breeze (they also kind of say badlands are so statiscally rare that we sould not care).
Anyway, in real job as a dev, when I made a moving average it was the most impressive math I done, and when I was resorting to ifft (convolution with a hammer function) I was told it should not be used because it was unmaintainable because no one understood.
I have been so ... hammered down by computer engineer so sure of themselves that I learnt to shut my mouth to keep my job.
Who really read this?
I love it, but I never did any serious brainy stuff as a dev.
Re: Practically Accurate Floating-Point Math
#29If you are interested in writing accurate numerical algorithms, as further reading I highly recommend reading "Accuracy and Stability of Numerical Algorithms" by Nick Higham, which is a very good comprehensive book. Like this article says, floating-point arithmetic is often thought of as mysterious, but it really isn't: it just obeys its own precisely specified rules. I think it's very good to dispel the mystery, so…
The first thing you should worry about is whether your computation is ill-conditioned, and only then worry whether it's numerically stable. OK, but how does one apply this advice in practice? The radiative exchange between two surfaces goes as x^4-y^4 where x and y are the surface temperatures. I can worry about whether that's ill-conditioned all I want, but at the end of the day I'm further ahead evaluating (x-y)(x+…
I don't know about your radiative exchange example, but it is often possible to reparametrize the problem. For example, sometimes you can replace the parameters (x,y) with (s,t)=(x-y, y), which would get you further: x^2-y^2 = s (s+2t), which is stable at s=0, and if x>y>0, then it's stable everywhere. This can shift the region of instability to somewhere far from typical parameter values. Otherwise, yes, this isn't solvable.
Re: Practically Accurate Floating-Point Math
#30Sorry to tell you, but 99% of so called expert developers don't understand this. For them a real number is a a float (exactly). And I have been fired for saying e-commerce should be done with fixed point numbers. (we do + - * / at most) Headaches really comes with multi currency web sites since conversion brings incommesurability. They also told me that since we are using linear equation most of the time (are we?), e…
Ugh. It's shameful when people use wrong arguments about mathematics to oppress. I really wish you the best.
This article first appeared in "Computing in Science and Engineering" magazine, so its audience is mainly scientists in various fields who use floating point in their research. Because of that breadth, I tried to make it as accessible as possible, while teaching enough floating-point principles to make debugging make sense.
I honestly hope that everyone who uses floating point reads this.