Live data from Hacker News

Beautiful branchless binary search

probablydance.com

11–20 of 198 posts

Re: Beautiful branchless binary search

#11
A cool related algorithm is https://algorithmica.org/en/eytzinger

In addition to being branchless, it also has better cache properties than a standard binary search tree.

If you're doing a binary search in an array, you start in the middle, and then jump all the way to the midway point of one half, and so on. As a result, there's a lot of distance between each read that you do in the array. Thus, each read is putting extra strain on the cache (or the reads won't be cached).

The CPU cache performs much better if the values you want to read are close to each other. Enter the Eytzinger Binary Search. The idea is that, if you're always going to be accessing the root of the tree, and then one of its two children--you should just put those children physically close to the root, so they'll all be on the same cache line!

Re: Beautiful branchless binary search

#12
post #8

Trusting the C++ compiler to emit a specific instruction for guaranteed performance - cute, but not realistic.

I don't think it is the instruction that makes the difference. Cmov is a branch either way. Branches can be unpredictable and slow, no matter what instructions, (or tables, or whatever) are used to introduce the branch at question. It is predictable nature of branching in this algo that makes the difference. Which makes me wonder... EDIT: the cmov vs explicit branching story is a bit more complicated than just branch…

Cmov doesn't branch. A branch refers specifically to the program counter ending up in more than one possible place after an instruction has executed. It is this behavior that mucks with the CPU state and slows everything down.

It's true that the cmov instruction uses the CPU flags register (which I'm sure the CPU designers at Intel hate), but that doesn't mean that it branches.

You can achieve the same effect as cmov by using bitwise operations and a subtraction, though it'd just be a few cycles slower--but it would be even more clear that it doesn't branch.

Re: Beautiful branchless binary search

#13

Nice algorithm. I don't agree that it is branchless however. Cmov is a branching instruction for sure. And it doesn't really matter whether a branch can be well predicted. The reason you mainly want branchless code is for security reasons where a timing sidechannel could reveal information. Calling this algorithm branchless devalues the term into something meaningless. Edit: Everybody in the comments is focusing on p…

> The reason you mainly want branchless code is for security reasons That just isn’t true.

Which part isn't true? I think the qualifier 'mainly' definitely weakens the statement, but I'm struggling to think of how it could certainly be false statement.

This is certainly true for at minimum DDoS attacks, and I would absolutely consider that a security risk.

Re: Beautiful branchless binary search

#14

Earlier quoted context omitted.

> The reason you mainly want branchless code is for security reasons That just isn’t true.

Which part isn't true? I think the qualifier 'mainly' definitely weakens the statement, but I'm struggling to think of how it could certainly be false statement. This is certainly true for at minimum DDoS attacks, and I would absolutely consider that a security risk.

When implementing cryptographic primitives, you want to avoid branching on secret values. The reason why is that the CPU's branch predictor will attempt to predict the value that you're branching on, and thus something about the values that you're branching on gets revealed if you can see how long a CPU takes to run a task/perform a function.

This is more than just a theoretical issue. These channel attacks have been demonstrated in practice, even if the victim CPU is running across the internet.

Re: Beautiful branchless binary search

#15
> Those spikes for std::lower_bound are on powers of two, where it is somehow much slower. I looked into it a little bit but can’t come up with an easy explanation

It’s been a long time since I looked into this, if I recall correctly binary search with close to power of 2 sized data is slower because of some interaction with the caches which causes the data to constantly have to be reloaded from memory.

Re: Beautiful branchless binary search

#17
I'm all for supporting OSS, but the author having a tip button with "recommended" tips of 20 $ and 1,000 $ for people and business respectively is so laughable it took away from an otherwise good blog posts. Commercialisation of OSS 2.0 should be discouraged to this extend

Re: Beautiful branchless binary search

#18
post #8

Earlier quoted context omitted.

I don't think it is the instruction that makes the difference. Cmov is a branch either way. Branches can be unpredictable and slow, no matter what instructions, (or tables, or whatever) are used to introduce the branch at question. It is predictable nature of branching in this algo that makes the difference. Which makes me wonder... EDIT: the cmov vs explicit branching story is a bit more complicated than just branch…

Cmov doesn't branch. A branch refers specifically to the program counter ending up in more than one possible place after an instruction has executed. It is this behavior that mucks with the CPU state and slows everything down. It's true that the cmov instruction uses the CPU flags register (which I'm sure the CPU designers at Intel hate), but that doesn't mean that it branches. You can achieve the same effect as cmov…

Sure, it doesn't change the PC. But it can introduce a branch indirectly, I.e. when there is a jump to an address MOVed by cmov.

Either way, it seems that the wisdom has change since the last time I wrote and read assembly. Cmov used to be slow. It seems that the current answer is "it depends".

Re: Beautiful branchless binary search

#19
post #8

Earlier quoted context omitted.

I don't think it is the instruction that makes the difference. Cmov is a branch either way. Branches can be unpredictable and slow, no matter what instructions, (or tables, or whatever) are used to introduce the branch at question. It is predictable nature of branching in this algo that makes the difference. Which makes me wonder... EDIT: the cmov vs explicit branching story is a bit more complicated than just branch…

Cmov doesn't branch. A branch refers specifically to the program counter ending up in more than one possible place after an instruction has executed. It is this behavior that mucks with the CPU state and slows everything down. It's true that the cmov instruction uses the CPU flags register (which I'm sure the CPU designers at Intel hate), but that doesn't mean that it branches. You can achieve the same effect as cmov…

The main problem with branches in an algorithm like this is that they create a dependency, where not only do the future instructions depend on the possibly-mispredicted outcome of the previous instruction and need to get thrown out, but you may have started executing the wrong instructions.

A cmov doesn't affect the instruction pointer, so that 'executing the wrong instructions' condition is gone, but it still affects the output of all following instructions if they depend on the register it modifies. So you still have to worry about pipelined work being thrown away on a mispredict.

IMO, cmov was far more valuable back when it was new, because branches were more expensive then. IIRC it was added to x86 in something like the MMX or SSE1 era, so a very long time ago for very different processor architectures. It's still a useful instruction category to have, if only because it lets you produce smaller code (and smaller code is usually faster), but I expect the value is less significant than it used to be.

Incidentally I am curious whether Linus's position on cmov back in 2007 is one he still holds today, and whether his assertions hold up on a modern Intel or AMD CPU. https://yarchive.net/comp/linux/cmov.html

I think the branchless search in this blog post is faster simply because the inner loop is much smaller. The observation that it sometimes does extra comparisons explains why it is slower for an expensive comparison operator, because the advantage of the smaller loop is gone. I don't think it being "branchless" is terribly important in comparison. You could try to fudge an experiment to prove this by inserting extra instructions, but they would need to be ones that decode into actual executed uops, and you'd need to decide whether they should have dependencies or not.

In the bad old days when I got promoted to the programming team at a job they had me take a test, and one of the tasks was to write a transparent blit routine. The one I wrote was "branchless", but because cmov intrinsics weren't widely available I achieved this by using an array of pointers and selecting the array index based on the result of a comparison, something like:

  unsigned char* src_dest[] = { src, dest };
  for (...; ...; src_dest[0]++, src_dest[1]++)
    *src_dest[1] = *src_dest[*src_dest[0] == transparent_value];
the lead who reviewed the test had to come by and ask me to explain how it worked. Certainly, this was "branchless", even without the presence of cmov, since it was just doing movs based on a 0/1 produced by the comparison. But the actual source of the dependency here is the comparison, not a jmp or a cmov. The loop was faster than a branch on the CPUs of that era but these days it would probably be much slower.

Re: Beautiful branchless binary search

#20

Nice algorithm. I don't agree that it is branchless however. Cmov is a branching instruction for sure. And it doesn't really matter whether a branch can be well predicted. The reason you mainly want branchless code is for security reasons where a timing sidechannel could reveal information. Calling this algorithm branchless devalues the term into something meaningless. Edit: Everybody in the comments is focusing on p…

> And it doesn't really matter whether a branch can be well predicted

I guess we build branch predictors on our CPUs for fun?

Post reply on HN