Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware of fast-math

simonbyrne.github.io

Post reply on HN
Print view View original

71–80 of 111 posts

Re: Beware of fast-math

#71

Post by [deleted] »

pavpanchekha wrote: post #38
named-user wrote: post #35

Earlier quoted context omitted.

The docs for fast math literally say this "This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications." So what did you expect? I was taught in my undergrad class…

Hi, I noticed you're a new user, so in that spirit this is not the kind of comment that is well-received on HN. It reads as arrogant and condescending, though I assume that was not your intention. Since I publish regularly in this area, the condescension feels especially off-base. To answer the question in good faith: while the documentation clearly states that fast-math can reduce accuracy, in isolation the accuracy…

[deleted]

Re: Beware of fast-math

#73

Post by SeanLuke »

titzer wrote: post #66
adrian_b wrote: post #28

Earlier quoted context omitted.

Flushing subnormals to zero produces speed gains only on certain CPU models, while on others it almost does not have any effect. For example Zen CPUs have negligible penalties for handling denormals, but many Intel models have a penalty between 100 and 200 clock cycles for an operation with denormals. Even on the CPU models with slow denormal processing, a speedup between 100 and 1000 exists only for the operation wi…

When we were debating whether WebAssembly should support subnormal numbers (i.e. be IEEE compliant), some people often cited these mythical subnormal slowdowns. So Dan Gohman ran some benchmarks and the scary-sounding slowdowns amounted to something like less than 1% (i.e. noise) for almost all benchmarks. Interestingly, one benchmark did not converge correctly with FTZ (i.e. no subnormals) and actually ran 3x more i…

> When we were debating whether WebAssembly should support subnormal numbers (i.e. be IEEE compliant), some people often cited these mythical subnormal slowdowns. So Dan Gohman ran some benchmarks and the scary-sounding slowdowns amounted to something like less than 1% (i.e. noise) for almost all benchmarks. Interestingly, one benchmark did not converge correctly with FTZ (i.e. no subnormals) and actually ran 3x more iterations, leading to a 3x slowdown.

I recently built a modular additive music synthesizer called Flow (https://github.com/eclab/flow). When certain modules in the synthesizer [gradually] push certain state variables into the denormal range, my synthesizer will experience a roughly 100x slowdown. Mind you, this isn't due to DSP or even sound processing, and Flow isn't written in C, but in 100% pure *Java*. Since Java can't turn off denormals, I have to manually check for and zero them at strategic locations to avoid getting mired in the denormal quicksand.

Re: Beware of fast-math

#74

Post by titzer »

SeanLuke wrote: post #73
titzer wrote: post #66

Earlier quoted context omitted.

When we were debating whether WebAssembly should support subnormal numbers (i.e. be IEEE compliant), some people often cited these mythical subnormal slowdowns. So Dan Gohman ran some benchmarks and the scary-sounding slowdowns amounted to something like less than 1% (i.e. noise) for almost all benchmarks. Interestingly, one benchmark did not converge correctly with FTZ (i.e. no subnormals) and actually ran 3x more i…

> When we were debating whether WebAssembly should support subnormal numbers (i.e. be IEEE compliant), some people often cited these mythical subnormal slowdowns. So Dan Gohman ran some benchmarks and the scary-sounding slowdowns amounted to something like less than 1% (i.e. noise) for almost all benchmarks. Interestingly, one benchmark did not converge correctly with FTZ (i.e. no subnormals) and actually ran 3x more…

Complain to Intel. AMD and ARM chips have no such 100x penalties.

Re: Beware of fast-math

#75

Post by couchand »

dahart wrote: post #69
an1sotropy wrote: post #37

Earlier quoted context omitted.

I remember a teacher who said (when I was a student) something like "if you care about precision use double". Now that I'm teaching, I force students to only use single-precision "float"s in their code, with the message that FP precision is a finite resource, and you don't learn how to manage any resource by increasing its supply. I think my students hate me.

Knowing said teacher ;) I wonder if he’d still say the same thing now… It’s good practice to have to use single precision (or even half-precision!) now and then in order to be forced to deal with precision issues. Yes, use doubles if you really need them and aren’t trying to learn. But they’re often a lot more than 2x more expensive, and they might not be necessary at all. I’ve heard people who develop commercial ren…

Perhaps you were in the same lecture as me, when I asked the lead developer on Big Hero 6 why they didn't just use doubles to solve their precision woes, and he informed me that they literally couldn't afford to use doubles at that scale.

Re: Beware of fast-math

#76

Post by SeanLuke »

titzer wrote: post #74
SeanLuke wrote: post #73

Earlier quoted context omitted.

> When we were debating whether WebAssembly should support subnormal numbers (i.e. be IEEE compliant), some people often cited these mythical subnormal slowdowns. So Dan Gohman ran some benchmarks and the scary-sounding slowdowns amounted to something like less than 1% (i.e. noise) for almost all benchmarks. Interestingly, one benchmark did not converge correctly with FTZ (i.e. no subnormals) and actually ran 3x more…

Complain to Intel. AMD and ARM chips have no such 100x penalties.

Perhaps true. But the point is: you're calling denormal failures "edge cases", yet my primary experience with denormals is exactly this.

Re: Beware of fast-math

#77

Post by titzer »

SeanLuke wrote: post #76
titzer wrote: post #74

Earlier quoted context omitted.

Complain to Intel. AMD and ARM chips have no such 100x penalties.

Perhaps true. But the point is: you're calling denormal failures "edge cases", yet my primary experience with denormals is exactly this.

I sympathize. Software/hardware is littered with one person's edge cases being another person's entire world. But in the grand scheme, yes, subnormals are exceedingly rare. Clearly Intel microarchitecture designers think that, as they seem perfectly willing to continue punishing some applications with a massive performance cliff. Their mitigation should never have been "we'll add a cheat switch for speed" but rather "we'll work as hard as our competitors do to make these cases fast." Standards are supposed to do that, but cheaters abound (and yes I am being a bit perjorative--cheaters don't think of themselves as cheating, they merely have important use cases that demand special dispensation).

GPU hardware is a different, but similar story, from what I can see. It saves transistors to do FTZ, and the originally niche usage of FP to put pixels on the screen didn't really care so much about niggling details. But GPUs became general purpose and important, and they've been dragged into full compliance by application demands. It's the only sane outcome in the end. Instead, all this FTZ stuff has just made a mess at layers above. It would all be unnecessary if subnormals were as fast as AMD, ARM, IBM, and other chip manufacturers have managed to make them.

Re: Beware of fast-math

#78

Post by mattpharr »

shoo wrote: post #64
pavpanchekha wrote: post #27

Fun fact: when working on Herbie ( http://herbie.uwplse.org ), our automated tool for reducing floating-point error by rearranging your mathematical expressions, we found that fast-math often undid Herbie's improvements. In a sense, Herbie and fast-math are opposites: one makes code more accurate (sometimes slower, sometimes faster), while the other makes code faster (sometimes less accurate, sometimes more).

thank you for sharing the link to Herbie, that looks like a useful tool. If I follow at high level, it looks like Herbie is trying to rewrite expressions to minimise error without runtime performance constraints. Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound? i guess compilers are generally focused on the latter problem, perhaps with…

> Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound?

There are! See followup work by @pavpanchekha and others on "Pherbie", which finds a set of Pareto-optimal rewritings of a program so that it's possible to trade-off error and performance: https://ztatlock.net/pubs/2021-arith-pherbie/paper.pdf.

Re: Beware of fast-math

#79

Post by shoo »

mattpharr wrote: post #78
shoo wrote: post #64

Earlier quoted context omitted.

thank you for sharing the link to Herbie, that looks like a useful tool. If I follow at high level, it looks like Herbie is trying to rewrite expressions to minimise error without runtime performance constraints. Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound? i guess compilers are generally focused on the latter problem, perhaps with…

> Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound? There are! See followup work by @pavpanchekha and others on "Pherbie", which finds a set of Pareto-optimal rewritings of a program so that it's possible to trade-off error and performance: https://ztatlock.net/pubs/2021-arith-pherbie/paper.pdf .

Fantastic! This appears to be available in the command line version of herbie with the `--pareto` flag:

> Enables multi-objective improvement. Herbie will attempt to simultaneously optimize for both accuracy and expression cost. Rather than generating a single "ideal" output expression, Herbie will generate many output expressions. This mode is still considered experimental. This will take a long time to run. We recommend timeouts measured in hours.

Re: Beware of fast-math

#80

Post by adgjlsfhk1 »

mattpharr wrote: post #78
shoo wrote: post #64

Earlier quoted context omitted.

thank you for sharing the link to Herbie, that looks like a useful tool. If I follow at high level, it looks like Herbie is trying to rewrite expressions to minimise error without runtime performance constraints. Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound? i guess compilers are generally focused on the latter problem, perhaps with…

> Are there alternative tools that focus on rewriting code to maximise performance while keeping error below some configurable bound? There are! See followup work by @pavpanchekha and others on "Pherbie", which finds a set of Pareto-optimal rewritings of a program so that it's possible to trade-off error and performance: https://ztatlock.net/pubs/2021-arith-pherbie/paper.pdf .

I've generally been disappointed by ferbie when I've tried it, but this does look really cool. I would love it if this provides a path to making it easier to get good tradeoffs between precision and accuracy.
Post reply on HN