Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone’s Been Messing with My Subnormals

moyix.blogspot.com

Post reply on HN
Print view View original

71–80 of 132 posts

Re: Someone’s Been Messing with My Subnormals

#71

Post by nsajko »

Athas wrote: post #54
nsajko wrote: post #48

Earlier quoted context omitted.

Well, how would you improve the docs? Both documentation entries seem reasonable to me. That said, I don't see why the -Ofast option even needs to exist, except backwards compatibility, as -ffast-math and the others can (and should IMO) be specified explicitly.

Consider the documentation for the similar compiler flag in the OpenCL specification: > -cl-unsafe-math-optimizations > Allow optimizations for floating-point arithmetic that (a) assume that arguments and results are valid, (b) may violate IEEE 754 standard and (c) may violate the OpenCL numerical compliance requirements as defined in section 7.4 for single-precision floating-point, section 9.3.9 for double-precision…

The relevant GCC man page entries are even more clear than the OpenCL spec excerpt.

-ffast-math:

> This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions.

It also point to the -funsafe-math-optimizations sub-option, where it is said that:

> Allow optimizations for floating-point arithmetic that (a) assume that arguments and results are valid and (b) may violate IEEE or ANSI standards. When used at link time, it may include libraries or startup files that change the default FPU control word or other similar optimizations. [...]

Re: Someone’s Been Messing with My Subnormals

#72

Post by nsajko »

brigade wrote: post #57
nsajko wrote: post #49

Earlier quoted context omitted.

You're wrong, both the doc entry for -Ofast and the one for -ffast-math say that they can result in incorrect programs . Programs are produced by linking, so I don't see what other way to interpret this is possible.

Why not simply replace all FP math with a constant zero? That’d be really fast and an equally valid strict interpretation of “can result in incorrect programs.”

See https://news.ycombinator.com/newsguidelines.html, e.g.:

> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

Re: Someone’s Been Messing with My Subnormals

#74

Post by brigade »

nsajko wrote: post #72
brigade wrote: post #57

Earlier quoted context omitted.

Why not simply replace all FP math with a constant zero? That’d be really fast and an equally valid strict interpretation of “can result in incorrect programs.”

See https://news.ycombinator.com/newsguidelines.html , e.g.: > Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

Just because you’re shallowly dismissing my comment doesn’t make it wrong.

Linking in code with undefined (in this case, redefined) behavior doesn’t automatically invalidate the entire program. But thats the language used because once the undefined behavior is hit at runtime, the spec no longer defines what the behavior is and what the program will do afterwards.

Re: Someone’s Been Messing with My Subnormals

#75

Post by moyix »

leni536 wrote: post #73

Does this only affect pypi, or should I now worry about shared libraries shipped with my distro as well? Debian is not crazy enough to ship shared libs compiled with -ffast-math, right? RIGHT?

Please don't do this to me, I don't know if I have it in me to go on ANOTHER big scrape & scan.

Re: Someone’s Been Messing with My Subnormals

#76

Post by magicalhippo »

Denormalized numbers is one reason why you really want to think carefully if you try to optimize code by rewriting expressions involving multiplication and division.

For example, if you got "x = (a / b) * (c / d)" one might think that rewriting it as "x = (a * c) / (b * d)" will save you a division and gain you speed. It will and it might, respectively.

However it will also potentially break an otherwise safe operation. If the numbers are very small, but still normal, then the product (b * d) might result in a denormalized number, and dividing by it will result in +/- infinity.

However, the code might guarantee that the ratios (a / b) and (c / d) are not too small or too large, so that multiplying them is guaranteed to lead to a useful result.

Re: Someone’s Been Messing with My Subnormals

#78

Post by eigenspace »

aidenn0 wrote: post #67
ChrisRackauckas wrote: post #22

The Julia package ecosystem has a lot of safeguards against silent incorrect behavior like this. For example, if you try to add a package binary build which would use fast math flags, it will throw an error and tell you to repent: https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/... In user codes you can do `@fastmath`, but it's at the semantic level so it will change `sin` to `sin_fast` but not recurse do…

Automatic FMA can change the result of operations, so it makes (some) sense to be bundled in with fastmath.

This isn't really as valid a comparison as you might think it is. The results of operations varying is not the problem with 'fast-math', the problem is that can negatively impact accuracy in catastrophic ways (among other things).

Sure, automatic FMA can change the result, but to my knowledge it always gives a more accurate result, not a less accurate one, and the way in which the results may differ is bounded.

Re: Someone’s Been Messing with My Subnormals

#79

Post by titzer »

cesarb wrote: post #32

At a previous company I worked at, we had an issue with our software (Windows-based, written in a proprietary language) randomly crashing. After some debugging, we found that this happened whenever the user made some specific actions, but only if, in that session, the user had previously printed something or opened a file picker. The culprit was either a printer driver or a shell extension which, when loaded, changed…

I've heard so many stories akin to this one that I just shake my head. It's a self-inflicted wound that people who prioritize performance above other considerations keep inflicting on everyone else.

I hope we learned our lessons on this specific question in the design of Wasm. There are subnormals in Wasm and you can't turn them off for performance.

Re: Someone’s Been Messing with My Subnormals

#80

Post by titzer »

jcranmer wrote: post #18

The problem here is that enabling FTZ/DAZ flags involves modifying global (technically thread-local) state that is relatively expensive to do. Ideally, you'd want to twiddle these flags only for code that wants to work in this mode, but given the relative expense of this operation, it's not entirely practicable to auto-add twiddling to every function call, and doing it manually is somewhat challenging because compile…

I don't think flipping these flags is expensive. Can you provide a source for that? AFAICT modern microarchitectures are going to register-rename that into the u-ops issued to the functional units, rather than flush the entire ROB.
Post reply on HN