Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Practically Accurate Floating-Point Math

computer.org

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Practically Accurate Floating-Point Math

#11

Post by stephencanon »

acqq wrote: post #8
stephencanon wrote: post #6

Earlier quoted context omitted.

The correct way to evaluate x^2 - 1 is by using fma(x,x,-1). Now that Intel and AMD have finally made FMA available in hardware on commodity parts (better late than never!), it's realistic to start using this much more freely.

The expression won't turn to be good conditioned only by using fma. You actually missed the point of conistonwater's post.

Condition number only matters if you assume that the inputs are perturbed by some error. Under the assumption that inputs are exact (which is appropriate to make for some uses), it does not enter into the analysis.

Re: Practically Accurate Floating-Point Math

#12

Post by conistonwater »

stephencanon wrote: post #10
conistonwater wrote: post #9

Earlier quoted context omitted.

That won't do much for x near 1, because the function x^2-1 is itself ill-conditioned. In other words, it is relevant that the floating-point value of x is itself only an approximation to some true value of x. So computing x^2-1 exactly for a given floating-point value of x does not give a good approximation to the true value of x^2-1. This is a mathematical property of the function x^2-1, and cannot be fixed with an…

There are multiple ways to analyze computations; condition number is one of them. It is relevant when inputs are assumed to be approximations to some hidden "correct" value. However, this assumption is not always warranted when dealing with floating-point numbers; sometimes, we would instead like to analyze errors under the assumption that the inputs are exact values. When this is the case, fma(x,x,-1) produces a cor…

Okay, I agree, that is completely reasonable. I do usually work with fairly complex algorithms, so I care about individual operations less than about the final result.

Re: Practically Accurate Floating-Point Math

#13

Post by ThatGeoGuy »

Probably an obvious error, but on page 81 (second page), they say the following:

    Another correct test case is as follows:
    > (rat (float -1 23 -6))
    - (26) / (34)
In truth, this should be - 23 / 64, which is a very different number. Perhaps there's comical value in an article about accurate math having inaccurate typography?

Re: Practically Accurate Floating-Point Math

#14

Post by malkia »

dvt wrote: post #4

Even though refreshing and enlightening, the article doesn't cover another (major) reason why floating-point math is generally avoided (especially in high-performance applications): computational slowdown when dealing with subnormals[1][2]. I feel that there is a lot of overlap between cases where you may want to minimize error while at the same time still be performant (simulations, ray tracing, rendering, etc.). So…

AFAIK gpu's and ps3 cell chip did not support subnormals for that reason, also as the other poster said, they can be disabled. Alternatively you can add white-noise to your data to avoid it (Say after an IIR filter on sound data).

Re: Practically Accurate Floating-Point Math

#15

Post by ntoronto »

ThatGeoGuy wrote: post #13

Probably an obvious error, but on page 81 (second page), they say the following: Another correct test case is as follows: > (rat (float -1 23 -6)) - (26) / (34) In truth, this should be - 23 / 64, which is a very different number. Perhaps there's comical value in an article about accurate math having inaccurate typography?

That is indeed a typo. Thanks for catching it.

It's amazing how much you miss when you think you already know what's written.

Re: Practically Accurate Floating-Point Math

#16

Post by ntoronto »

alricb wrote: post #7
ntoronto wrote: post #2

This article (like all other IEEE Computing Now articles) is free to download for a limited time.

Is it me, or have all the ells (the letter L) been replaced by ones (the digit 1) in the PDF? Edit: It's me. It's just a font with sloping serifs, which is what confused me.

It's not necessarily just you. I thought it was wrong when I first opened it. There may be something funny going on with the font - I think the "1" and "l" are switched.

Edit: Never mind, it copies/pastes correctly. The font is just different from what I'm used to.

Re: Practically Accurate Floating-Point Math

#17

Post by dalke »

conistonwater wrote: post #3

If you are interested in writing accurate numerical algorithms, as further reading I highly recommend reading "Accuracy and Stability of Numerical Algorithms" by Nick Higham, which is a very good comprehensive book. Like this article says, floating-point arithmetic is often thought of as mysterious, but it really isn't: it just obeys its own precisely specified rules. I think it's very good to dispel the mystery, so…

"Use (x-y)(x+y)" is part of the "folk wisdom." It's between "We even know some folk wisdom intended to avoid the dangers." and "or is it increasing magnitude?", where the last bit shows that perhaps these aren't the best solutions.

Re: Practically Accurate Floating-Point Math

#18

Post by conistonwater »

dalke wrote: post #17
conistonwater wrote: post #3

If you are interested in writing accurate numerical algorithms, as further reading I highly recommend reading "Accuracy and Stability of Numerical Algorithms" by Nick Higham, which is a very good comprehensive book. Like this article says, floating-point arithmetic is often thought of as mysterious, but it really isn't: it just obeys its own precisely specified rules. I think it's very good to dispel the mystery, so…

"Use (x-y)(x+y)" is part of the "folk wisdom." It's between "We even know some folk wisdom intended to avoid the dangers." and "or is it increasing magnitude?", where the last bit shows that perhaps these aren't the best solutions.

No, later in the article they recommend using it, saying that it decreases floating-point errors.

Re: Practically Accurate Floating-Point Math

#19

Post by dalke »

conistonwater wrote: post #18
dalke wrote: post #17

Earlier quoted context omitted.

"Use (x-y)(x+y)" is part of the "folk wisdom." It's between "We even know some folk wisdom intended to avoid the dangers." and "or is it increasing magnitude?", where the last bit shows that perhaps these aren't the best solutions.

No, later in the article they recommend using it, saying that it decreases floating-point errors.

Ahh! Yes, in the box on p94 titled "About That Folk Wisdom". Thanks for the correction!

Re: Practically Accurate Floating-Point Math

#20

Post by alricb »

ntoronto wrote: post #16
alricb wrote: post #7

Earlier quoted context omitted.

Is it me, or have all the ells (the letter L) been replaced by ones (the digit 1) in the PDF? Edit: It's me. It's just a font with sloping serifs, which is what confused me.

It's not necessarily just you. I thought it was wrong when I first opened it. There may be something funny going on with the font - I think the "1" and "l" are switched. Edit: Never mind, it copies/pastes correctly. The font is just different from what I'm used to.

TIL: Prestige Elite [1] might not be the best font out there for programming.

1: http://en.wikipedia.org/wiki/Prestige_Elite

Post reply on HN