Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of Fast-Math

simonbyrne.github.io

Post reply on HN
Print view View original

221–230 of 233 posts

Re: Beware of Fast-Math

#221

Post by kolbe »

djrj477dhsnv wrote: post #136
kolbe wrote: post #119

Earlier quoted context omitted.

All your price field messages are sent to the exchange and back via fixed point, so you are using fixed point for at least some of the process (unless you're targeting those few crypto exchanges that use fp prices). If you need to be extremely fast (like fpga fast), you don't waste compute transforming their fixed point representation into floating.

Sure, string encodings are used for most APIs and ultra HFT may pattern match on the raw bytes, but for regular HFT if you're doing much math, it's going to be floating point math.

We might have different definitions of "HFT"

Re: Beware of Fast-Math

#222

Post by pie_flavor »

Athas wrote: post #158
pie_flavor wrote: post #123

Earlier quoted context omitted.

One of the things I always appreciate about the crypto community is that you do not have to ask what numeric type is being used for money, it is always 8-digit fixed-point. No floating-point rounding errors to be found anywhere.

How does this avoid rounding error? Division and multiplication and still result in nonrepresentable numbers, right?

It is not hard to remember what int division is about, when your types are ints in code. It also comes up almost never, and isn't what floating-point rounding error means. You aren't multiplying money 99% of the time, and when you are, you don't care about exacting precision (e.g. 20% discount). Floating-point rounding error, on the other hand, is about how 0.1 + 0.2 != 0.3.

Re: Beware of Fast-Math

#224

Post by [deleted] »

kbolino wrote: post #207
kccqzy wrote: post #106

Earlier quoted context omitted.

You will not lose a cent here and there just by using float64, for the range of values that banks deal with. For added assurance, just round to the nearest cent after each operation.

You can round to the nearest .0078125 (1/128) but not to the nearest .01 (1/100) because that number cannot be represented in float64. To round to the nearest cent, you would need to make cents your units (i.e. the quantity "1 dollar" would be represented as 100 instead of 1.0).

[deleted]

Re: Beware of Fast-Math

#225

Post by kccqzy »

kbolino wrote: post #218
nurettin wrote: post #208

Earlier quoted context omitted.

Within the general context of this discussion, "cannot be represented" is a red herring. You don't need to have a representation of the exact number 0.1 if you can tolerate errors after the 7th decimal (and it turns out you can). And 0.1+0.1+0.1 does not have to be comparable with 0.3 using operator==. You have an is_close function for that. And accumulation is not an issue because you have rounding and fma for that.

Ok, but in the context of this thread, it is important. Remember that I'm replying to the statement "For added assurance, just round to the nearest cent after each operation." That is misleading advice and the behavior of floats is just context for why. First of all, a lot of languages don't include arbitrary rounding in their math libraries at all , only having rounding to integers. Second, in the docs of Python, wh…

> First of all, a lot of languages don't include arbitrary rounding in their math libraries at all, only having rounding to integers.

You do the simple, obvious and correct thing: multiply by 100, round to int, convert to double, divide by 100. It does not matter whether the final division by 100 results in an exact value. (You might argue this is inefficient but it's not a correctness problem.)

> for example, round(2.675, 2) gives 2.67 instead of the expected 2.68.

You are not going to execute round(2.675, 2) if you follow my advice of rounding after every operation. Because the error will never reach 0.005. Your argument is moot.

Re: Beware of Fast-Math

#226

Post by Dylan16807 »

anthk wrote: post #214
Dylan16807 wrote: post #212

Earlier quoted context omitted.

Every implementation of floating point can handle small to medium integers without rounding. So that example doesn't show anything we can learn from.

Buth for Forth you can have a totally working software floating point made from the people whose later would set the IEEE standard. Think about very small microcontrollers without FP in hardware, you can be sure that a software FP implementation will perfectly work under the standard thresolds.

Okay, I guess? I appreciate having a well-verified library when I can't have floating point hardware, but I'm never in that situation and I really don't see how it's relevant to this discussion. Any floating point implementation is going to have rounding problems, including IEEE. The example above is IEEE.

Re: Beware of Fast-Math

#227

Post by kbolino »

kccqzy wrote: post #225
kbolino wrote: post #218

Earlier quoted context omitted.

Ok, but in the context of this thread, it is important. Remember that I'm replying to the statement "For added assurance, just round to the nearest cent after each operation." That is misleading advice and the behavior of floats is just context for why. First of all, a lot of languages don't include arbitrary rounding in their math libraries at all , only having rounding to integers. Second, in the docs of Python, wh…

> First of all, a lot of languages don't include arbitrary rounding in their math libraries at all, only having rounding to integers. You do the simple, obvious and correct thing: multiply by 100, round to int, convert to double, divide by 100. It does not matter whether the final division by 100 results in an exact value. (You might argue this is inefficient but it's not a correctness problem.) > for example, round(…

You can certainly encounter 2.675 as a multiplier, even if you wouldn't have it as a balance.

It doesn't matter that some error starts off way less than 0.005 if rounding then amplifies it. We can find two 2-digit numbers that multiply to get exactly 2.675 in reals, but whose product differs from the float number closest to 2.675 enough to affect rounding:

    abs(      2.14 * 1.25     -       2.675    )  0.01
And regarding integer vs fractional rounding, we can see different results for what is nominally the same computation, depending on where the decimal point is:

    abs(round(1.0  * 1.5, 0) - 2.0 )  0.001
Now, I never said that floats were bad. I am only saying that rounding them doesn't work the way one might expect, and shouldn't be done any more than necessary; in many cases, it's not necessary at all.

Re: Beware of Fast-Math

#228

Post by StefanKarpinski »

goalieca wrote: post #57
Affric wrote: post #36

Earlier quoted context omitted.

How does IEEE 754 prevent auto-vectorisation?

Floating point arithmetic is neither commutative or associative so you shouldn’t.

Floating-point arithmetic is non-associative, but it is commutative for the operations that are algebraically commutative: x + y == y + x and x*y == y*x. And x - y = -(y - x) so subtraction is properly anti-commutative.

The only very marginal exception to this is that when both arguments are NaN, the return value will be NaN, but which NaN payload is returned can depend on argument order. But no one ever uses this because it's not specified, so it can't be used reliably for anything useful. The behavior I wish IEEE 754 had specified for this is to define a standard NaN value (or two), and when the return value of an op is NaN, and some of the arguments are non-standard NaNs, then one of those non-standard NaN values must be returned. This doesn't depend on argument order and allows NaN payloads to be reliably propagated, which would let you encode useful debugging information in NaN payloads and know that it will flow through the program.

Re: Beware of Fast-Math

#229

Post by rlpb »

CamouflagedKiwi wrote: post #188
rlpb wrote: post #22

Earlier quoted context omitted.

> The optimisations in -O3 aren't supposed to give incorrect results. I didn't mean to imply that they result in incorrect results. > they make a more aggressive space/speed tradeoff... Right...so "better" becomes subjective, depends on the use case, so it doesn't make sense to choose -O3 blindly unless you understand the trade-offs and want that side of them for the particular builds you're doing. Things that everyo…

If they're things that everyone wants, why aren't they in -O1?

-O1 is defined not to perform "any optimizations that take a great deal of compilation time". See the definitions in the manpage.

Re: Beware of Fast-Math

#230

Post by stingraycharles »

anthk wrote: post #87
stingraycharles wrote: post #48

Earlier quoted context omitted.

Yup, because of the imprecision of floating points, cannot just assume that “(a + c) + (b + d)” is the same as “a + b + c + d”. It would be pretty ironic if at some point fixed point / bignum implementations end up being faster because of this.

They are, just check anything fixed-point for the 486SX vs anything floating under a 486DX. It's faster scaling and sum and print the desired precision than operating on floats.

Is that also the case for modern architectures? Eg is there SIMD fixed precision?
Post reply on HN