Practically Accurate Floating-Point Math • Hacker News Board

Re: Practically Accurate Floating-Point Math

#31

Post by ntoronto » Tue, Oct 28, 2014, 11:20 AM UTC

stephencanon wrote: post #5

dvt wrote: post #4

Even though refreshing and enlightening, the article doesn't cover another (major) reason why floating-point math is generally avoided (especially in high-performance applications): computational slowdown when dealing with subnormals[1][2]. I feel that there is a lot of overlap between cases where you may want to minimize error while at the same time still be performant (simulations, ray tracing, rendering, etc.). So…

A couple points: Support for subnormals can easily be disabled (usually be enabling "Flush to Zero" behavior) to allow for better performance on essentially all recent hardware. Over the last few years, subnormal stalls have shrunk rapidly, and gone away entirely in some cases. Intel started this process with the Sandybridge architecture. These stalls are also greatly reduced in some arm64 implementations.

We had this discussion on the Racket user's mailing list a couple of years ago. One user with an AMD processor experienced significant stalls; others with Intel processors didn't.

I don't know the status of AMD's floating point at the moment, but I hope they're doing something about that.

Re: Practically Accurate Floating-Point Math

#32

Post by ntoronto » Tue, Oct 28, 2014, 11:34 AM UTC

wukix wrote: post #25

ntoronto wrote: post #23

Earlier quoted context omitted.

All correct, for exact rationals. Racket's exact rational arithmetic tends to take about 1000x the amount of time floating-point arithmetic takes to compute similar functions, and creates a lot of garbage on the heap. It gets worse, though: with long-running computations, exact rationals' numerators and denominators tend to grow without bound, unless you explicitly round them to a fixed precision. If you take the lat…

Wu-Decimal uses exact rationals (they aren't base 10 floats). Division works according to normal Lisp semantics, since the CL ratio type is used for arithmetic. Let's say you divide something by 3 and now you have infinitely repeating digits: then it is no longer in set D, and Wu-Decimal no longer considers it to be of decimal type. Instead, it is treated as a fraction, again per standard CL semantics. The "Printing"…

I've monkeyed with a lot of numeric representations, and every one of them involves tough theoretical and practical trade-offs. Returning an exact rational as the result of division sounds reasonable.

My favorite representation so far pairs a signed float in [2^-512,2^512) with a signed integer to extend the exponent range. The idea is to avoid overflow and underflow, particularly when multiplying thousands of probabilities.

(On average, adding a few thousand log probabilities or densities and then exponentiating the sum yields a number with about 9 digits precision. Worst case for adding log probabilities and exponentiating is about 8 digits precision, and for adding log densities it's 0 digits. Multiplying the same number of probabilities or densities retains about 13 digits in the worst case.)

Re: Practically Accurate Floating-Point Math

#33

Post by SFjulie1 » Wed, Oct 29, 2014, 1:08 AM UTC

ntoronto wrote: post #30

SFjulie1 wrote: post #28

Sorry to tell you, but 99% of so called expert developers don't understand this. For them a real number is a a float (exactly). And I have been fired for saying e-commerce should be done with fixed point numbers. (we do + - * / at most) Headaches really comes with multi currency web sites since conversion brings incommesurability. They also told me that since we are using linear equation most of the time (are we?), e…

Ugh. It's shameful when people use wrong arguments about mathematics to oppress. I really wish you the best. This article first appeared in "Computing in Science and Engineering" magazine, so its audience is mainly scientists in various fields who use floating point in their research. Because of that breadth, I tried to make it as accessible as possible, while teaching enough floating-point principles to make debuggi…

Computer science on the field is ugly.

I went in university to learn physics, so I have my geometry and numerical analysis basic understanding.

I honestly thinks people who learnt it has a cursus for graduating did not care, and that they actually don't understand their difficulties even though they learnt about it. They don't see it is a float.

One of the other misconception I have to fight with is what is time.

CS engineer believe in a perfect synchronized time and timestamp that are monotonicly growing the same everywhere with a 10-9 precision and rely on that for distributed systems. My guts (and general relativity too I guess) are telling me they are wrong.

But since I am stuck reimplementing a bubble sort for phone numbers I don't quite have the time to work on it. I have to live with my tingling intuitions telling me I am either an overpessimistic ass being a pain for my colleague that is becoming obsolete by lack of studying, or the whole industry seems getting more magic oriented than science oriented.

Anyway, like a lot of people I am getting inadapted out there.

At least, I have spare time to work on game of life and complex systems... sometimes, and I have a lovely caring wife, so I am happy either way :)