Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frame pointers vs. DWARF – my verdict

rwmj.wordpress.com

Post reply on HN
Print view View original

11–20 of 67 posts

Re: Frame pointers vs. DWARF – my verdict

#11

Post by sylware »

dwarf = brain convolution

frame pointer = 1 less register on an already set of registers under heavy strain (x86_64). I do envy risc-v with all their registers: even the load-store model of risc-v won't use enough more registers to end up with the same amount of strain as on x86_64.

If I ever have the usage of dwarf, I would build the tables manually only for what I wish to debug?... if it is possible (without the use of specialized assembler directives, because it increase significantly the technical cost of the assembler). That said, does a real, accurate and complete specification of dward exists? Because when I look at the sysv ABI or ELF, what a mess.

Re: Frame pointers vs. DWARF – my verdict

#12

Post by masklinn »

sylware wrote: post #11

dwarf = brain convolution frame pointer = 1 less register on an already set of registers under heavy strain (x86_64). I do envy risc-v with all their registers: even the load-store model of risc-v won't use enough more registers to end up with the same amount of strain as on x86_64. If I ever have the usage of dwarf, I would build the tables manually only for what I wish to debug?... if it is possible (without the us…

> frame pointer = 1 less register on an already set of registers under heavy strain (x86_64).

X86-64 is not under heavy register pressure strain. That idea is a legacy from x86 (plain, 32b).

x86-64 has the same register count as ARMv7 and few bothered disabling the frame pointer there, even though it’s a load-store architecture.

Re: Frame pointers vs. DWARF – my verdict

#13

Post by nneonneo »

The article points out that the kernel uses ORC instead of DWARF for unwinding. I wonder if that could ever become an option in userspace? I imagine that if all you’re interested in is stack traces, instead of debugging (which DWARF is designed for), ORC would be a very nice performance win. And it’s not as if they’re mutually exclusive, either: there’s no reason why a binary or debuginfo couldn’t just ship both.

In any case, the runtime performance hit of frame pointers is quite high, and since we do have things like DWARF (and maybe ORC someday?), I’d still argue that frame pointers aren’t necessary. It’s nice to have that extra register free!

Re: Frame pointers vs. DWARF – my verdict

#14

Post by the_mitsuhiko »

conconpon wrote: post #10
the_mitsuhiko wrote: post #7

Earlier quoted context omitted.

> To summarize, unwinding via frame pointers does not miss any information that would be collected with DWARF unwinding. Everything can be recovered later at symbolization time. The real issue with DWARF based unwinding/symbolication is that it's really complex on the user experience around it. We at Sentry support stack walking from minidumps, yet we often cannot unwind on Linux platforms on the server because execu…

Couldn't this be solved by uploading the binaries along with the crash dumps, if you don't already have a copy of it, as determined by checking hashes or something?

Exactly, except debuginfod from Canonical and some others misses the executables.

Re: Frame pointers vs. DWARF – my verdict

#15

Post by DaveFlater »

Uh... [Begin flashback] 2012: A change made in GCC 4.7 allowed optimization to reschedule and defer the push of the frame pointer that previously occurred in the function prologue whenever frame pointers were enabled. When binaries are profiled using frame pointers, incorrect call chains are derived whenever a sample is taken between the top of the function and the instruction that pushes the frame pointer. I complained, but got an immediate WONTFIX: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55667

So did they fix that eventually or is everyone just oblivious?

How I found the problem in 2012: Configuration of profiling tools for C/C++ applications under 64-bit Linux https://doi.org/10.6028/NIST.TN.1790

Re: Frame pointers vs. DWARF – my verdict

#16

Post by titzer »

> But collecting the whole stack would consume far too much storage, so by default it only collects the first 8K. Many userspace stacks will be larger than this, in which case the data collection will simply be incomplete – it will never be possible to recover the full stack trace.

Yeah, doing an 8KB memcopy for every profile sample sounds like a lot of overhead. Is DWARF unwinding so slow that that is actually faster?

Virgil doesn't use a frame pointer, and I sort of regret it. It uses custom unwinding information that is used during GC or throwing an exception (i.e. controlled crash). I spent considerable time optimizing both the space and time of that lookup, to the point where it's only 32 bits of metadata per call site and a few dozen instructions to walk each frame. But that was a major, major pain to debug and I found a bug in it at late as last year.

A frame pointer is also required for stack allocation of objects that aren't fully scalar-replaceable. Virgil doesn't do that yet, but aspires to someday.

Re: Frame pointers vs. DWARF – my verdict

#17

Post by amluto »

nneonneo wrote: post #13

The article points out that the kernel uses ORC instead of DWARF for unwinding. I wonder if that could ever become an option in userspace? I imagine that if all you’re interested in is stack traces, instead of debugging (which DWARF is designed for), ORC would be a very nice performance win. And it’s not as if they’re mutually exclusive, either: there’s no reason why a binary or debuginfo couldn’t just ship both. In…

Native ORC generation from GCC or LLVM or userspace tooling like objtool would be nifty.

FWIW, ORC was specifically designed to be efficient, but it was not designed to be future-proof against complex toolchain changes. Since all the ORC tooling is in the kernel, it can evolve together if needed.

(I was a bit involved in the design — I helped optimize the format to reduce cache misses on lookup.)

Re: Frame pointers vs. DWARF – my verdict

#18

Post by amluto »

rwmj wrote: post #5
dezgeg wrote: post #2

Nice analysis. It would be interesting to see comparison to Intel LBR. Also would be nice to know how profiling unwinding is done on Windows (maybe someone knows how to summon Bruce Dawson).

AIUI the problems with LBR are two-fold. It only works on newish CPUs, and it only handles a limited number of stack frames (I heard 8, but maybe more on recent CPUs).

CET’s shadow stacks, on the other hand, will solve this entire problem, both exactly and extremely efficiently.

Re: Frame pointers vs. DWARF – my verdict

#19

Post by tialaramex »

masklinn wrote: post #12
sylware wrote: post #11

dwarf = brain convolution frame pointer = 1 less register on an already set of registers under heavy strain (x86_64). I do envy risc-v with all their registers: even the load-store model of risc-v won't use enough more registers to end up with the same amount of strain as on x86_64. If I ever have the usage of dwarf, I would build the tables manually only for what I wish to debug?... if it is possible (without the us…

> frame pointer = 1 less register on an already set of registers under heavy strain (x86_64). X86-64 is not under heavy register pressure strain. That idea is a legacy from x86 (plain, 32b). x86-64 has the same register count as ARMv7 and few bothered disabling the frame pointer there, even though it’s a load-store architecture.

Right, x86-64 offers eight extra register names† (r8 through r15). If you choose to go from x86-without-frame-pointer to x86-64-with-frame-pointer you gained 7 register names which is huge.

This makes the case where that one extra register name makes all the difference much rarer, arguably turning it from "I demand a compiler flag" to "Let's just hand-write the machine code for this one very special routine if our performance data suggests it's worth it".

† Internally a modern CPU has far more actual register, to enable a feature called "register renaming". But we can only talk about them using their canonical names, and x86-64 adds eight more of those.

Re: Frame pointers vs. DWARF – my verdict

#20

Post by the_mitsuhiko »

tialaramex wrote: post #19
masklinn wrote: post #12

Earlier quoted context omitted.

> frame pointer = 1 less register on an already set of registers under heavy strain (x86_64). X86-64 is not under heavy register pressure strain. That idea is a legacy from x86 (plain, 32b). x86-64 has the same register count as ARMv7 and few bothered disabling the frame pointer there, even though it’s a load-store architecture.

Right, x86-64 offers eight extra register names† (r8 through r15). If you choose to go from x86-without-frame-pointer to x86-64-with-frame-pointer you gained 7 register names which is huge. This makes the case where that one extra register name makes all the difference much rarer, arguably turning it from "I demand a compiler flag" to "Let's just hand-write the machine code for this one very special routine if our pe…

I think generally the talk about "there are not enough registers" ignore pipelining and register renaming way too much. The loss of performance of the frame pointer register even on x86 is not that problematic, and on x86_64 it's completely negligible unless you're in a tight switch heavy interpreter loop.
Post reply on HN