Live data from Hacker News

Profiling with Ctrl-C

yosefk.com

21–30 of 42 posts

Re: Profiling with Ctrl-C

#21

I mostly use GUI-based debuggers (and profilers), but even in this case I found it often useful to pause the program at random times when it appears "stuck". Most of the time I don't event need to reach for a profiler proper.

Yes, exactly, I do this all the time. Only after having exhausted this, which I consider to be the low-hanging fruits of performance gains, do I start profiling code with an actual profiler.

Re: Profiling with Ctrl-C

#22
post #7

My favorite hack along these lines was to put a timer/ISR on an embedded system that did nothing more than crawl up the stack frame the two or three addresses that the ISR used (yep, it was really just as dumb as [sp + 8] or whatever), and then dump that address to the serial terminal every second or so. You can fix a lot of stupid problems that way. (And most problems are stupid.) Yes, yes, a real profiler would be…

> Yes, yes, a real profiler would be better As someone who wrote several profilers for a living... that is a real profiler.

only once you pipe the output to sort | uniq -c

Re: Profiling with Ctrl-C

#23

I mostly use GUI-based debuggers (and profilers), but even in this case I found it often useful to pause the program at random times when it appears "stuck". Most of the time I don't event need to reach for a profiler proper.

Random sampling is not only useful for quick and dirty debugging, but also for engineering nuclear bombs: https://en.wikipedia.org/wiki/Metropolis–Hastings_algorithm#...

few phenomena in daily life are either quicker or dirtier than a nuclear bomb

Re: Profiling with Ctrl-C

#24
post #13
post #11

Speaking of keyboard shortcuts, I miss BSD's Ctrl-T and SIGINFO. It often helped to see if a process was hung.

I don't know exactly what these BSD things did, but there is a super easy way nowadays to get the stack for any process: eu-stack -i -p $(pidof ...) Thanks to debuginfod this will even give you good backtraces right away (at the cost of some initial delay to load the data from the web, consecutive runs are fast). If you get a "permission denied" error, you probably need to tweak kernel.yama.ptrace_scope=0

the bsd things still work; you can install a bsd in qemu or a spare laptop and try them

from your reference to kernel.yama.ptrace_scope (and your apparent belief that bsd belongs to the distant past) i infer that eu-stack is a linux thing? this looks pretty awesome, thanks for the tip!

https://stackoverflow.com/questions/12394935/getting-stacktr...

Re: Profiling with Ctrl-C

#25

My favorite hack along these lines was to put a timer/ISR on an embedded system that did nothing more than crawl up the stack frame the two or three addresses that the ISR used (yep, it was really just as dumb as [sp + 8] or whatever), and then dump that address to the serial terminal every second or so. You can fix a lot of stupid problems that way. (And most problems are stupid.) Yes, yes, a real profiler would be…

It's amusing; your 2nd paragraph drives at the core of the programmer experience - having to disclaim everything you do and say with "yes, I know there are better ways, I'm not dumb, I'm just working within constraints, and this does the job".

Re: Profiling with Ctrl-C

#26

> Apparently gcc generates some DWARF data that gdb is slow to handle. The GNU linker fixes this data, so that gdb doesn’t end up handling it slowly. LLD refuses to emulate this behavior of the GNU linker, because it’s gcc’s fault to have produced that DWARF data in the first place. And gdb refuses to handle LLD’s output efficiently, because it’s LLD’s fault to not have handled gcc’s output the way the GNU linker doe…

I do get moderately annoyed by having to write code that's fundamentally a workaround for somebody else's failure, though I usually still do it anyway.

Sometimes I do so but add something to the stderr output referencing the issue number I'm compensating for - that has a surprisingly good rate of getting somebody who knows what they're doing looking at the issue in the other project and submitting a patch.

Re: Profiling with Ctrl-C

#27
post #15

https://poormansprofiler.org/

The premise of this website and articles like https://yosefk.com/blog/how-profilers-lie-the-cases-of-gprof... just show that the authors are using the wrong tools. It is nowadays relatively easy to also look at off-CPU time when profiling with perf (e.g. https://github.com/KDAB/hotspot/?tab=readme-ov-file#off-cpu-... ). The idea is to use sampling for the on-CPU periods and then combine that with the off-CPU time mea…

kreinin spends a lot of time debugging things that don't run on linux or any cpu architecture linux or vtune supports. even on amd64 linux, perf is not so useful with python, lua, node.js, browser js, shell scripts, etc.

Re: Profiling with Ctrl-C

#28
post #20

Earlier quoted context omitted.

> The premise of this website and articles like https://yosefk.com/blog/how-profilers-lie-the-cases-of-gprof ... just show that the authors are using the wrong tools. It is nowadays relatively easy to also look at off-CPU time when profiling with perf (e.g. https://github.com/KDAB/hotspot/?tab=readme-ov-file#off-cpu- ...). I think, firstly, that spending 15s trying the CTRL-c approach is a worthwhile tradeoff. If you…

perf is easily available through yocto and buildroot (and probably other embedded linux image builders). hotspot can be downloaded as an appimage. It should not take 30-60min to set this up, but granted, learning the tools the first time always has some cost. Furthermore, note how your reasoning is quite different from what the website you linked to says - it basically says "there are no good tools" (which is untrue)…

the vast majority of embedded cpus cannot run yocto or indeed linux, even the arms

but they all support gdb

Re: Profiling with Ctrl-C

#29

> what do you know, there’s one billion stack frames from the nlohmann JSON parser, I guess it all gets inlined in the release build; My guess would be that it's because tail-call optimisation only happens in -O2 and above. Parsing recursively is frequently the cleanest way to implement a parser of tree-structured input, after all. If you're doing anything recursively, it makes sense to slightly restructure the recur…

It looks to me like you can use -foptimize-sibling-calls to get it to happen on gcc below -O2.

There's also https://github.com/pietro/gcc-musttail-plugin to ensure it does happen (and clang has musttail support built in these days).

Post reply on HN