Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Someone’s Been Messing with My Subnormals

moyix.blogspot.com

Post reply on HN
Print view View original

61–70 of 132 posts

Re: Someone’s Been Messing with My Subnormals

#61

Post by mananaysiempre »

moyix wrote: post #36
mananaysiempre wrote: post #31

Following the article’s links, I fail to find an actual example of anything failing to converge in flush-subnormals mode. I mean, I’m sure one could be squeezed out, but the justification given amounts to “Sterbenz’s lemma [the one that rephrases “catastrophic cancellation” as “exact differences”] fails, maybe something somewhere also will”. And my (shallow but not nonexistent) experience with numerical analysis is t…

The gevent issue has an example: https://github.com/gevent/gevent/pull/1820 I haven't examined the code of scipy.stats.skellam.sf so I can't say for sure that it's not converging, but it's clearly some kind of pathological behavior.

So somebody tried to calculate, for integer arguments from 0 to 99 inclusive, the CDF of the difference of two Poisson variables with means 4e-6 and 1e-6? I... don’t know if it is at all reasonable to expect an answer to that question. As in, genuinely don’t know—obviously it’s an utterly rotten thing to compute, but at the same time maybe somebody got really interested in that and figured out a way to make it work.

Anyhow, my spelunking was cut off by sleep, so the best I can tell that would end up in the CDFLIB[1] routine CUMCHN with X = 8e-6, PNONC = 2e-6, DF from 0 to 99. The insides don’t really look like the kind of magic that is held up by Sterbenz’s lemma and strategically arranged to take advantage of gradual underflow, so at first glance I wouldn’t trust anything subnormal-dependent that it would compute, but maybe it still is? Sleep.

[1] https://people.sc.fsu.edu/~jburkardt/f_src/cdflib/cdflib.f90

Re: Someone’s Been Messing with My Subnormals

#62

Post by jeffbee »

yjftsjthsd-h wrote: post #51
jerf wrote: post #41

Earlier quoted context omitted.

It's also a bit less relevant when everything is so fast. I used Gentoo on a cheap-for-the-time Pentium 133MHz. Gentoo was basically the difference between a modestly pleasant system and an unusably slow system if I tried to run a standard still-compiled-for-386 distro on it. I've long since stopped worrying about it because on the systems I run, which are not top-of-the-line but aren't RPis either, it's not worth wo…

Yeah, I don't know the breakdown between better hardware and better compiler optimizations (even in the default settings) and less differentiation between processors, but I've done some minor not-very-scientific tests of compiling packages with O3/march=mtune=native and in my limited experience it wasn't particularly useful. Like, not just small benefits, but zero or below the noise floor benefits in my benchmarks. O…

Tune for native sometimes makes a difference but not always. Targeting a platform that is known to have AVX2, instead of detecting AVX2 at runtime and bouncing through the PLT, can make a large difference. PGO remains the largest opportunity.

Re: Someone’s Been Messing with My Subnormals

#63

Post by speeder »

pavlov wrote: post #40
klysm wrote: post #34

Earlier quoted context omitted.

That is one hell of a war story - I didn't realize that kind of failure was even possible, but it is truly terrifying.

Direct3D used to flip the x87 FPU to single precision mode by default. This produced some amazing bugs when your other C libraries reasonably assumed that a double would be at least 64 bits. (The FPU mode settings affected the thread that called Direct3D, and most programs used to be single-threaded.) It seems they changed this behavior in Direct3D 10: https://microsoft.public.win32.programmer.directx.graphics.n...

I stumbled into this bug in a rather spetacular manner.

I was making a game using D3D, Lua and Chipmunk physics, and some of the behaviour of the game was being odd.

So I started to try printing random stuff with Lua, eventually I just tried: print(5+5), and to my surprise my console outputted "11".

I went into Lua's irc channel to talk about this, and everyone said I was nuts, that the number was too small to trigger precision issues, that I was a troll and so on.

After a lot of searching I found out about this D3D bug, so I switched the game to use OpenGL instead there it was, 5+5 = 10 again!

Now why fiddling with the FPU could make 5+5 become 11, I have no idea.

Re: Someone’s Been Messing with My Subnormals

#64

Post by speeder »

Night_Thastus wrote: post #42
moyix wrote: post #24

Earlier quoted context omitted.

Nope, it still links in crtfastmath: $ gcc -Ofast -fno-unsafe-math-optimizations -fpic -shared foo.c -o foo.so $ objdump -j .text --disassemble=set_fast_math foo.so foo.so: file format elf64-x86-64 Disassembly of section .text: 0000000000001040 : 1040: f3 0f 1e fa endbr64 1044: 0f ae 5c 24 fc stmxcsr -0x4(%rsp) 1049: 81 4c 24 fc 40 80 00 orl $0x8040,-0x4(%rsp) 1050: 00 1051: 0f ae 54 24 fc ldmxcsr -0x4(%rsp) 1056: c3…

Ouch. Two flags that should reasonably stop this, and neither do. This feels a bit like the time I was told "No, -wAll does not in fact add all warnings".

Wait, it doesn't? O.o

Re: Someone’s Been Messing with My Subnormals

#65

Post by RodgerTheGreat »

mrtesthah wrote: post #55

I thought the purpose of Python was to make development simple and predictable. Needing to track down the compilation and linker flags of every single shared library reveals the fallacy of this abstraction.

If a language wishes to reap the rewards of a pre-existing ecosystem, it must pay for the warts and misfeatures of that ecosystem. Python is deeply dependent on C libraries to achieve acceptable performance, and this is the price.

Re: Someone’s Been Messing with My Subnormals

#67

Post by aidenn0 »

ChrisRackauckas wrote: post #22

The Julia package ecosystem has a lot of safeguards against silent incorrect behavior like this. For example, if you try to add a package binary build which would use fast math flags, it will throw an error and tell you to repent: https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/... In user codes you can do `@fastmath`, but it's at the semantic level so it will change `sin` to `sin_fast` but not recurse do…

Automatic FMA can change the result of operations, so it makes (some) sense to be bundled in with fastmath.

Re: Someone’s Been Messing with My Subnormals

#68

Post by moyix »

mananaysiempre wrote: post #61
moyix wrote: post #36

Earlier quoted context omitted.

The gevent issue has an example: https://github.com/gevent/gevent/pull/1820 I haven't examined the code of scipy.stats.skellam.sf so I can't say for sure that it's not converging, but it's clearly some kind of pathological behavior.

So somebody tried to calculate, for integer arguments from 0 to 99 inclusive, the CDF of the difference of two Poisson variables with means 4e-6 and 1e-6? I... don’t know if it is at all reasonable to expect an answer to that question. As in, genuinely don’t know—obviously it’s an utterly rotten thing to compute, but at the same time maybe somebody got really interested in that and figured out a way to make it work.…

Yeah, unfortunately I have no idea if that was their original goal (which seems unlikely?) or if this is just a minimal example they came up with after tripping over the actual problem in a more realistic setting.

I think it suffices to show that the behavior of FTZ/DAZ caused an actual problem for someone, though. I agree that the vast majority of numerical code won't care about FTZ/DAZ, but when it's enabled thread-wide you have no idea what kind of code you'll end up affecting.

Re: Someone’s Been Messing with My Subnormals

#69

Post by nsajko »

mrguyorama wrote: post #53
nsajko wrote: post #48

Earlier quoted context omitted.

Well, how would you improve the docs? Both documentation entries seem reasonable to me. That said, I don't see why the -Ofast option even needs to exist, except backwards compatibility, as -ffast-math and the others can (and should IMO) be specified explicitly.

The fact that -ffast-math makes no mention that it will poison any other code executing in your process space is a huge missing point of info. Docs as written, anyone not doing scientific math should have that flag, but the reality is that most people have some code somewhere in their process that expects fairly sane floating point math behavior, even if it's just displaying progress bars or something.

> The fact that -ffast-math makes no mention that it will poison any other code executing in your process space

Untrue. The doc entry for -ffast-math says "can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions". Emphasis mine.

So they clearly say that the entire program can turn invalid when -ffast-math is used.

You and some other people here act like the docs say "translation unit" or something like that, instead of "program", but this is simply not the case.

Furthermore, the entry for -ffast-math points to entries for suboptions that -ffast-math turns on (located right below in the man page), e.g. -funsafe-math-optimization. These also make clear how dangerous they can be even when turned on one at a time.

Re: Someone’s Been Messing with My Subnormals

#70

Post by moyix »

speeder wrote: post #64
Night_Thastus wrote: post #42

Earlier quoted context omitted.

Ouch. Two flags that should reasonably stop this, and neither do. This feels a bit like the time I was told "No, -wAll does not in fact add all warnings".

Wait, it doesn't? O.o

Nope. clang has "-Weverything", and gcc has "-Wextra", both of which go beyond "-Wall".

https://stackoverflow.com/questions/11714827/how-can-i-turn-...

Post reply on HN