Earlier quoted context omitted.
For something like this, you can just take several trials and look at the minimum observed time, which is when there will have been ~no interruptions. https://github.com/facebook/folly/blob/main/folly/docs/Bench...
You don't actually know that for sure. You have only placed a new upper bound.
A 40-line fix eliminated a 400x performance gap
41–50 of 81 posts
Re: A 40-line fix eliminated a 400x performance gap
#42Re: A 40-line fix eliminated a 400x performance gap
#43Re: A 40-line fix eliminated a 400x performance gap
#44It took seven years to address this concern following the initial bug report (2018). That seems like a lot, considering how instrumenting CPU time can be in the hot path for profiled code.
400x slower than 70ns is still only 28us. How often is the JVM calling this function?
This is what flame graphs are super helpful for, to see whether it’s really a problem or not.
Also, remember that every extra moment running instructions is a lost opportunity to put the CPU to sleep, so this has energy efficiency impact as well.
Re: A 40-line fix eliminated a 400x performance gap
#45Flamegraphs are wonderful. Me: looks at my code. "sure, ok, looks alright." Me: looks at the resulting flamegraph. "what the hell is this?!?!?" I've found all kinds of crazy stuff in codebases this way. Static initializers that aren't static, one-line logger calls that trigger expensive serialization, heavy string-parsing calls that don't memoize patterns, etc. Unfortunately some of those are my fault.
Also cool that when you open it in a new tab, the svg [0] is interactive! You can zoom in by clicking on sections, and there's a button to reset the zoom level. [0]: https://questdb.com/images/blog/2026-01-13/before.svg
https://github.com/brendangregg/FlameGraph
Useful site if you are on to perf/eBPF/performance things with many examples and descriptions even for other uses as e.g. memory usage, disk usage (prefer heatmaps here but they are nice if you want to send someone a interactive view of their directory tree ...).
Re: A 40-line fix eliminated a 400x performance gap
#46Flamegraphs are wonderful. Me: looks at my code. "sure, ok, looks alright." Me: looks at the resulting flamegraph. "what the hell is this?!?!?" I've found all kinds of crazy stuff in codebases this way. Static initializers that aren't static, one-line logger calls that trigger expensive serialization, heavy string-parsing calls that don't memoize patterns, etc. Unfortunately some of those are my fault.
I also like icicle graphs for this. They're flamegraphs, but aggregated in the reverse order. (I.e. if you have calls A->B->C and D->E->C, then both calls to C are aggregated together, rather than being stacked on top of B and E respectively. It can make it easier to see what's wrong when you have a bunch of distinct codepaths that all invoke a common library where you're spending too much time.) Regular flamegraphs…
Re: A 40-line fix eliminated a 400x performance gap
#47Earlier quoted context omitted.
400x slower than 70ns is still only 28us. How often is the JVM calling this function?
It depends. If you’re doing continuous profiling, it’d make a call to get the current time at every method entry and exit, each of which could then add a context switch. In an absolute sense it appears to be small, but it could really add up. This is what flame graphs are super helpful for, to see whether it’s really a problem or not. Also, remember that every extra moment running instructions is a lost opportunity t…
Re: A 40-line fix eliminated a 400x performance gap
#48Earlier quoted context omitted.
That's a brilliant trick. The setup overhead and permission requirements for perf_event might be heavy for arbitrary threads, but for long-lived threads it looks pretty awesome! Thanks for sharing!
Yes you need some lazy setup in thread-local state to use this. And short-lived threads should be avoided anyway :)
Re: A 40-line fix eliminated a 400x performance gap
#49clock_gettime() goes through vDSO, avoiding a context switch. It shows up on the flamegraph as well.
Only for some clocks (CLOCK_MONOTONIC, etc) and some clock sources. For VIRT/SCHED, the vDSO shim still has to invoke the actual syscall. You can't avoid the kernel transition when you need per-thread accounting.
Re: A 40-line fix eliminated a 400x performance gap
#50Very interesting read.