Performance optimization is hard because it's fundamentally a brute-force task
1–10 of 153 posts
Re: Performance optimization is hard because it's fundamentally a brute-force task
#2An interesting aspect is data dependencies. If your next statement reuses data you just computed, that can cause pipeline bubbles, as that result you want to use just isn't available yet. I dived into that topic for a video about relative performance of old PCs I just published today.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#3I think it is worth making a distinction between "micro" (what the blogpost is about) and "macro", or "tactical" and "strategic", optimisations.
Strategic optimisations is often basically free if you have domain expertise. It's that easy to know that the business wants x outcome and algorithm y is the right choice etc if its all internal thought processes. Whereas if you don't know enough then you're likely to make very expensive to undo decisions.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#4An interesting aspect is data dependencies. If your next statement reuses data you just computed, that can cause pipeline bubbles, as that result you want to use just isn't available yet. I dived into that topic for a video about relative performance of old PCs I just published today.
Yes, there is non-obvious structure in some algorithms solely for the purpose of turning a single logical stream of dependent instructions into multiple concurrent streams of dependent instructions running through the same pipeline. The caveat of doing this, of course, is that it typically increases register pressure.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#5For low level benchmarking papi https://icl.utk.edu/papi/ is great tool. It provides access for counters like cache misses, branch misses and number of cycles.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#6I think it is worth making a distinction between "micro" (what the blogpost is about) and "macro", or "tactical" and "strategic", optimisations. Strategic optimisations is often basically free if you have domain expertise. It's that easy to know that the business wants x outcome and algorithm y is the right choice etc if its all internal thought processes. Whereas if you don't know enough then you're likely to make v…
I often refer to those as architectural optimizations. Even some of these tend to sensitive to the details of the operating environment.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#7When it comes to micro optimisations the issue is partly our usual tools in algorithm analysis and hardware intuition are very far apart. Random accessing memory is very slow compared to linear, some branches are considerably worse than others and it's actually quite hard to predict how much you can improve the low level details before you start, especially for big changes. Our tools can show us where time is being lost in inefficiencies but can't help us predict how changes will improve things.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#8The hardest bugs are the ones that only show up after you “optimize.”
Re: Performance optimization is hard because it's fundamentally a brute-force task
#9Good article that I agree with mostly. One interesting note is this:
> There is no way to provide both optimized assembly and equivalent C code and let the compiler use the former in the general case and the latter in special cases.
This is true, but can be seen as a failure of language and tooling. For example, Halide [1] pioneered (AFAIK) the concept of separating algorithm from implementation at the language level. This separation lets you express the algorithm once, and then "schedule" it by specifying parallelism, vectorization, etc. You can provide multiple schedules for one algorithm, which allows you to specialize / make different choices depending on varying factors.
It's a really interesting concept, though maybe limited in practice to DSLs. I'm not sure a general purpose language would be a good fit for this model, but then, for general purpose programs written in general purpose languages, perf optimization at the level TFA discusses is frequently limited to just specific hot sections. Those hot sections could be extracted out into specialized components written in such a DSL.
Re: Performance optimization is hard because it's fundamentally a brute-force task
#10Good article that I agree with mostly. One interesting note is this: > There is no way to provide both optimized assembly and equivalent C code and let the compiler use the former in the general case and the latter in special cases. This is true, but can be seen as a failure of language and tooling. For example, Halide [1] pioneered (AFAIK) the concept of separating algorithm from implementation at the language level…
gcc supports function multiversioning:
https://gcc.gnu.org/onlinedocs/gcc/Function-Multiversioning....