Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone’s Been Messing with My Subnormals

moyix.blogspot.com

Post reply on HN
Print view View original

81–90 of 132 posts

Re: Someone’s Been Messing with My Subnormals

#81

Post by UncleEntity »

mananaysiempre wrote: post #61
moyix wrote: post #36

Earlier quoted context omitted.

The gevent issue has an example: https://github.com/gevent/gevent/pull/1820 I haven't examined the code of scipy.stats.skellam.sf so I can't say for sure that it's not converging, but it's clearly some kind of pathological behavior.

So somebody tried to calculate, for integer arguments from 0 to 99 inclusive, the CDF of the difference of two Poisson variables with means 4e-6 and 1e-6? I... don’t know if it is at all reasonable to expect an answer to that question. As in, genuinely don’t know—obviously it’s an utterly rotten thing to compute, but at the same time maybe somebody got really interested in that and figured out a way to make it work.…

My last bug report I wrote a small C++ program to put all the values between 0x000 .. 0xfff into a tree structure and then iterate over the tree printing out the values.

I’d have loved if the library author replied with “why don’t you just print out the values directly?”

Re: Someone’s Been Messing with My Subnormals

#82

Post by bee_rider »

magicalhippo wrote: post #76

Denormalized numbers is one reason why you really want to think carefully if you try to optimize code by rewriting expressions involving multiplication and division. For example, if you got "x = (a / b) * (c / d)" one might think that rewriting it as "x = (a * c) / (b * d)" will save you a division and gain you speed. It will and it might, respectively. However it will also potentially break an otherwise safe operati…

Anyway, since there aren't any dependencies between a, b, c, and d, I would expect the two divisions to end up basically in parallel in the pipeline. So the critical path is a division and a multiplication either way. Of course that is just a guess.

Re: Someone’s Been Messing with My Subnormals

#83

Post by [deleted] »

jeroenhd wrote: post #7
puffoflogic wrote: post #2

Dynamic linking is the root of all kinds of evil, enough said.

What is the alternative here? To provide a python.so file with all possible binary Python packages statically linked into it? You'd need to update it every hour to include all the bugfixes in every native library yanked in! To recompile Python itself every time you install a package? Even with a compiler cache you'd have the Gentoo experience of waiting for ages every time you try to use the package manager. Dynamic…

[deleted]

Re: Someone’s Been Messing with My Subnormals

#85

Post by astrange »

ack_complete wrote: post #45
cesarb wrote: post #32

At a previous company I worked at, we had an issue with our software (Windows-based, written in a proprietary language) randomly crashing. After some debugging, we found that this happened whenever the user made some specific actions, but only if, in that session, the user had previously printed something or opened a file picker. The culprit was either a printer driver or a shell extension which, when loaded, changed…

Had to deal with this same issue when I had a program supporting plugins, DLLs compiled with Delphi would turn on all the floating point traps. Took a while to track down what was causing FP faults in comctl32.dll. It got so bad that I had to put in a popup dialog that would name and shame the offending DLL so the authors would fix their broken plugins. It's an ABI violation in Windows since the ABI specifically defi…

You could also get an issue with x87/MMX where floating point code wouldn't work if you wrote some MMX code and didn't do an `emms` instruction afterward.

This is basically the reason compiler autovectorization doesn't do MMX.

Re: Someone’s Been Messing with My Subnormals

#86

Post by JonChesterfield »

leni536 wrote: post #73

Does this only affect pypi, or should I now worry about shared libraries shipped with my distro as well? Debian is not crazy enough to ship shared libs compiled with -ffast-math, right? RIGHT?

If the package build scripts from upstream have that in them, Debian packaged versions probably do too

Re: Someone’s Been Messing with My Subnormals

#87

Post by ChrisRackauckas »

aidenn0 wrote: post #67
ChrisRackauckas wrote: post #22

The Julia package ecosystem has a lot of safeguards against silent incorrect behavior like this. For example, if you try to add a package binary build which would use fast math flags, it will throw an error and tell you to repent: https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/... In user codes you can do `@fastmath`, but it's at the semantic level so it will change `sin` to `sin_fast` but not recurse do…

Automatic FMA can change the result of operations, so it makes (some) sense to be bundled in with fastmath.

But if what you want is automatic FMA, then why carry along every other possible behavior with it? Just because you want FMA, suddenly NaNs are turned into Infs, subnormal numbers go to zero, handling of sin(x) at small values is inaccurate, etc? To me that's painting numerical handling in way too broad of strokes. FMA also only increases numerical accuracy, it doesn't decrease numerical accuracy, so bundling it with unsafe transformations makes one uncertain now whether it has improved or decreased accuracy.

For reference, to handle this well we use MuladdMacro.jl which is a semantic transformation that turns x*y+z into muladd expressions, and it does not recurse into functions so it does not change the definitions of the callers inside of the macro scope.

https://github.com/SciML/MuladdMacro.jl

This is something that will always increase performance and accuracy (performance because muladd in Julia is an FMA that is only applied if hardware FMA exists, effectively never resorting to a software FMA emulation) because it's targeted to do only a transformation that has that property.

Re: Someone’s Been Messing with My Subnormals

#88

Post by magicalhippo »

bee_rider wrote: post #82
magicalhippo wrote: post #76

Denormalized numbers is one reason why you really want to think carefully if you try to optimize code by rewriting expressions involving multiplication and division. For example, if you got "x = (a / b) * (c / d)" one might think that rewriting it as "x = (a * c) / (b * d)" will save you a division and gain you speed. It will and it might, respectively. However it will also potentially break an otherwise safe operati…

Anyway, since there aren't any dependencies between a, b, c, and d, I would expect the two divisions to end up basically in parallel in the pipeline. So the critical path is a division and a multiplication either way. Of course that is just a guess.

That assumes you can do multiple divisions in parallel. Back in the good old days, a single division unit was the norm, and it still is on most microcontrollers (assuming they even have hardware floating-point division[1]).

Anyone have any references on how the current state of affairs on modern AMD/Intels?

[1]: ARM Cortex-M4 for example can have a hardware FPU, but where division and sqrt are optional, see https://developer.arm.com/documentation/102832/latest/

Re: Someone’s Been Messing with My Subnormals

#89

Post by astrange »

nsajko wrote: post #21

-Ofast isn't a good name for the option, but in GCC's defense the manual is pretty clear about all this, and there's no excuse for blindly turning on compiler options - they literally change the semantics of your code.

Naming options (or methods or projects) "fast" or "faster" is dangerous; either they're broken, or some day they'll turn out to not be fast anymore.

IIRC Swift changed -Ofast to -Ounchecked after I complained about it.

Re: Someone’s Been Messing with My Subnormals

#90

Post by mananaysiempre »

UncleEntity wrote: post #81
mananaysiempre wrote: post #61

Earlier quoted context omitted.

So somebody tried to calculate, for integer arguments from 0 to 99 inclusive, the CDF of the difference of two Poisson variables with means 4e-6 and 1e-6? I... don’t know if it is at all reasonable to expect an answer to that question. As in, genuinely don’t know—obviously it’s an utterly rotten thing to compute, but at the same time maybe somebody got really interested in that and figured out a way to make it work.…

My last bug report I wrote a small C++ program to put all the values between 0x000 .. 0xfff into a tree structure and then iterate over the tree printing out the values. I’d have loved if the library author replied with “why don’t you just print out the values directly?”

My point wasn’t that the program in the bug report did not do anything meaningful (duh). My point was that most numerical algorithms, once they hit intermediate underflow in the form of subnormals for a set of inputs, do not proceed to compute a result that is in any way related to their stated purpose (with the notable exception of string conversions). Unless a hard limit on the number of iterations is imposed, most iterative algorithms can also be made to hang by giving them unfortunate inputs. In both cases it is the responsibility of the numerical analyst invoking them to not provide such inputs.

So it seems likely (if not certain) that the bug report in question says, essentially, that a function inside scipy, given certain arguments, returns one sequence of meaningless bits quickly when gevent is not loaded and a different sequence of meaningless bits very slowly when it is. While that is certainly not nice, and I’d accept it as a bug in gevent, it still does not count as a piece of code succeeding at a reasonable task with subnormals but failing without them.

(I do not have any ideological opposition to such code—it would certainly be interesting to see some!)

Post reply on HN