Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your code is fast if you're lucky

tiki.li

Post reply on HN
Print view View original

21–30 of 90 posts

Re: Your code is fast if you're lucky

#21

Post by adgjlsfhk1 »

IshKebab wrote: post #6

Wow that is quite surprising. Almost seems like it could be a compiler bug tbh. Very fragile optimisation if not!

it's arguably more of a cpu bug than a computer bug. The problem is that predictable data determined whether a cmov or a branch is faster. cmov is only faster than if when the branch is unpredictable. Summer the compiler doesn't know what values your program will be called with, it can only pick and hope. To fix this, cpus could have an instruction like cmov but that learns whether speculation would be profitable and converts to treat it like a branch if better.

Re: Your code is fast if you're lucky

#22

Post by xlii »

marginalia_nu wrote: post #20
xlii wrote: post #14

Is it only me..? Quicksort is supposed to be an algorithm that has O(n) to O(n²) performance and O(n log n) being only an average performance case. Test was made on random data coming from different archs (so I doubt it's characteristic would be remotely identical). Given input size of 50M it means that performance could be between 50M (5e7) up to 2.5e15. That's like performance instability of 8 orders of magnitude.…

Looking at big-O isn't very informative. We have plenty of statistical tools for telling whether there is an effect even with noisy data.

I ran 10k test locally on 2e5 and I'm seeing 4 orders of magnitude instability, but very high local stability (i.e. runs within specific second are very stable, showing almost no deviation, runs couple second later are the same, but results are within 1 OoM of the prior results (smaller batches, 500 tests).

I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set.

Re: Your code is fast if you're lucky

#23

Post by SkiFire13 »

adgjlsfhk1 wrote: post #19
xlii wrote: post #14

Is it only me..? Quicksort is supposed to be an algorithm that has O(n) to O(n²) performance and O(n log n) being only an average performance case. Test was made on random data coming from different archs (so I doubt it's characteristic would be remotely identical). Given input size of 50M it means that performance could be between 50M (5e7) up to 2.5e15. That's like performance instability of 8 orders of magnitude.…

if you have decent (randomized) pivoting, you never hit the worst case or anything like it

You don't need randomized pivoting for this, there are deterministic ones like median of median that will also result in a O(nlogn) worst case.

Also note that with a randomized pivoting you _might_ hit a O(n^2) worst case, it's just that it's incredibly rare and cannot be forced by an attacker controlling your input, so for most practical purposes can be ignored.

Re: Your code is fast if you're lucky

#24

Post by cogman10 »

marginalia_nu wrote: post #20
xlii wrote: post #14

Is it only me..? Quicksort is supposed to be an algorithm that has O(n) to O(n²) performance and O(n log n) being only an average performance case. Test was made on random data coming from different archs (so I doubt it's characteristic would be remotely identical). Given input size of 50M it means that performance could be between 50M (5e7) up to 2.5e15. That's like performance instability of 8 orders of magnitude.…

Looking at big-O isn't very informative. We have plenty of statistical tools for telling whether there is an effect even with noisy data.

It's a good rule of thumb that can be quite useful without additional analysis. It's not always the right way to do performance tuning, but I can't count the number of times I've changed an O(n^3) to an O(n) and seen massive performance gains as a result.

Re: Your code is fast if you're lucky

#26

Post by marginalia_nu »

cogman10 wrote: post #24
marginalia_nu wrote: post #20

Earlier quoted context omitted.

Looking at big-O isn't very informative. We have plenty of statistical tools for telling whether there is an effect even with noisy data.

It's a good rule of thumb that can be quite useful without additional analysis. It's not always the right way to do performance tuning, but I can't count the number of times I've changed an O(n^3) to an O(n) and seen massive performance gains as a result.

Yeah it helps make awful code decent, and some algorithms are better than others, but in terms of high performance code, locality, vectorization, and branching often matter much more big O.

Re: Your code is fast if you're lucky

#27

Post by tux3 »

vlovich123 wrote: post #16
tux3 wrote: post #13

Earlier quoted context omitted.

You could read compiler books, but I would actually recommend reading about CPUs and computer architecture directly. If you understand how the hardware works, then the optimizations are all very natural and fit into the picture perfectly, instead of being some arcane compiler magic that you have to take as a disconnected fact. Personally I actually haven't read too many books on optimizations, I just absorbed informa…

I also agree that computer architecture is more important - it grounds your understanding of how to write efficient code regardless of platform since most machines today share very similar ideas (OOO execution, caches, NUMA etc). How ever, I will disagree slightly that all the optimizations compilers do are about optimizing for a given architecture; some transformations are just weird algorithmic black magic about op…

Right. You won't learn from a computer architecture book or a uarch guide about SSA form, or LICM, or other famous compiler principle like the central role of inlining decisions ("the mother of all optimizations"). I don't have a good resource to recommend here, this is where my lack of formal training bites. "Go read a hundred blog articles by compiler experts" doesn't feel like very useful advice.

>Knowing how to make sure the compiler sees through a given construct to give you the low level expression you want is too much art and randomness; we need better ways to express optimization expectations so that if the compiler fails to match expectations it becomes a loud compiler error.

There's a parallel with hardware there. Verilog is a kind of hardware language designed for an abstract simulator, in the same way than C is designed for a standard abstract machine for the sake of portability. You end up with an idea of the assembly/RTL you want the compiler/synthetizer to generate in your head, and then it's a game of writing the right pattern that will be recognized and generate the output you want.

I think this is partially unavoidable, because we're inherently asking the compiler to generate a non-portable target-specific output in what is supposed to be a portable high-level language. If you start injecting compiler hints or requirements in your "portable" code, it all becomes a bit of a mess. Part of the problem is also that the high-level languages we're using were designed at a time were many questions were still unsettled. Things like signed integers being two's complements is a recent change in C and C++. But I think some of it is intrinsic impedance mismatch between high-level code and machine code.

I'm not sure I would like a proliferation of annotations that direct exactly how the compiler should optimize (like "must use cmov/csel here"), because if internal optimizer choices become public API, people will rely on internals in their large legacy codebases. I expect this would be a force that ossifies the compiler and prevent optimizations from improving. The "register" and "inline" keywords in C used to mean something to the compiler. But they were misused, having them be a requirement would have held back performance more than anything.

Then again I accepted the same justification against Postgres planner hints, and now that the idea has been recast as a plan stability feature I'm actually very happy with that idea. I'm uncomfortable with letting old calcified codebases hold back compiler internal, but at the same time once you find a way to have the compiler generate what you want, there's a real need to not have it break silently when you upgrade.

Re: Your code is fast if you're lucky

#28

Post by delamon »

adgjlsfhk1 wrote: post #21
IshKebab wrote: post #6

Wow that is quite surprising. Almost seems like it could be a compiler bug tbh. Very fragile optimisation if not!

it's arguably more of a cpu bug than a computer bug. The problem is that predictable data determined whether a cmov or a branch is faster. cmov is only faster than if when the branch is unpredictable. Summer the compiler doesn't know what values your program will be called with, it can only pick and hope. To fix this, cpus could have an instruction like cmov but that learns whether speculation would be profitable and…

You can, in fact, give compiler a hint wether the branch will be predictable or not. E.g. clang has __builtin_unpredictable().

Re: Your code is fast if you're lucky

#29

Post by marginalia_nu »

xlii wrote: post #22
marginalia_nu wrote: post #20

Earlier quoted context omitted.

Looking at big-O isn't very informative. We have plenty of statistical tools for telling whether there is an effect even with noisy data.

I ran 10k test locally on 2e5 and I'm seeing 4 orders of magnitude instability, but very high local stability (i.e. runs within specific second are very stable, showing almost no deviation, runs couple second later are the same, but results are within 1 OoM of the prior results (smaller batches, 500 tests). I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over ran…

You don't happen to be running `srand(time(0))`?

> I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set.

But yeah, this is correct. When optimizing, we want to pin every variable other than the change, as much as possible.

Re: Your code is fast if you're lucky

#30

Post by cogman10 »

jdw64 wrote: post #8
khuey wrote: post #7

Earlier quoted context omitted.

I'm not sure it's a "technique" but the general insight worth taking away from this is that compiler authors often write optimizers to recognize specific patterns so writing your code in a more idiomatic form increases the odds an optimizer will be able to optimize it. In this specific instance, at the hardware level it helps to understand how the branch predictor works and why quicksort in particular is essentially…

About that kind of 'technique', I guess I should make it a habit to dig into the compiler, which is still a black box to me. I should study a few techniques myself. Have a good day

While the article is interesting, it's the case of a missed optimization by the compiler and (possibly) something that a future version of the compiler will catch.

This sort of optimization is one that'd I'd not spend too much time trying to fix or catch, unless you are doing it for fun or you have a specific piece of code in a hot path that needs to go fast.

Where I'd spend time if I were trying to write very fast code which a compiler is unlikely to get right is SIMD optimizations. Specifically with floating point values.

One thing compilers can't and won't do is reorganize floating point optimizations (well, unless you explicitly give them permission to do that). That means the way you write your floating point code can really nerf performance and exclude you from much faster assembly.

The sure fire way to actually make such code faster is learning and using SIMD expressions. The compilers can sometimes get this right, but it's quite fragile.

Post reply on HN