What are you looking for that is not currently part of the profile guided optimization capabilities of current compilers? It has been the case for years that you can compile a program with profiling/tracing hooks, run it through a representative workload, and feed the profiler output back to the compiler to get a faster executable. That's why no Firefox you compile yourself will outperform the one you download from M…
In my experience, none change the source-code themselves.
I find it really weird that the article says: > It’s curious that we have low cache hit rates, a lot of time stalled on cache/memory, and low bandwidth utilization. That's typical for most workloads! Software is almost never compute or bandwidth bound in my experience, but instead spends most of its time waiting on memory in pointer chasing code. This is especially true for code written in managed languages like Java…
It's curious that there's low bandwidth utilization despite the low but rates / cpu stalled waiting for memory. Don't you think? Perhaps random access of small data causes frequent waits without utilizing the bandwidth in a way that block copies would.
You don't get high bandwidth utilization by pointer chasing unless you have many threads doing it and you switch threads while waiting on memory. That's true for GPUs, not for typical server workloads running on CPUs.
I've occasionally wondered how long it takes highly optimized C/C++ to be surpassed by optimizing compilers due to CPU advancement and the optimizations either making compiler optimization harder, or the optimizations target assumptions about CPU architecture that are no longer valid. That is, what is the shelf life of a very low level CPU optimization for Intel hardware.
Well while strictly not on topic, there was this talk recently on micro-optimisations. https://www.youtube.com/watch?v=nXaxk27zwlk
While it seemed to cover more of the compiler optimizations and how to do some low level benchmarking and optimizing and wasn't really addressing when those might become obsolete, it was really interesting and informative. Thanks!
A lot of those optimizations would no longer yield any benefits[0]. The CPU archictecture evolved a lot in 16 years, especially in branch/code prediction to the point where a correctly predicted branch (without branch_likely) has almost no cost. [0]: At least, this is true for x86 CPUs.
As a CPU architect, I can confirm that all those except possibly 2) will not yield significant benefits. Prefetching hints will only be useful when the particular code fragment is highly memory-bound because most wide superscalar microarchitectures will easily hide L1/L2 miss latencies.
My qp trie code " rel="nofollow">http://dotat.at/prog/qp/> got a performance boost of about 20% by adding prefetch hints in the obvious places. The inner loop is fetch / compute / fetch / compute, chaining down into the true. The next fetch will (usually) be some small offset from a pointer we can get immediately after its preceding fetch, so prefetch the base pointer, then compute to work out the exact offset.
A lot of those optimizations would no longer yield any benefits[0]. The CPU archictecture evolved a lot in 16 years, especially in branch/code prediction to the point where a correctly predicted branch (without branch_likely) has almost no cost. [0]: At least, this is true for x86 CPUs.
I've occasionally wondered how long it takes highly optimized C/C++ to be surpassed by optimizing compilers due to CPU advancement and the optimizations either making compiler optimization harder, or the optimizations target assumptions about CPU architecture that are no longer valid. That is, what is the shelf life of a very low level CPU optimization for Intel hardware.
There's been an interesting talk on this a few months ago: