Live data from Hacker News

Fastest branchless binary search

mhdm.dev

11–20 of 155 posts

Re: Fastest branchless binary search

#11

The Zig stdlib does not call out to C++ for binary search. The binary search is currently here: https://github.com/ziglang/zig/blob/b835fd90cef1447904d3b009... (edit: fixed link to not decay, thanks)

Versioned link: https://github.com/ziglang/zig/blob/b835fd90cef1447904d3b009...

Re: Fastest branchless binary search

#12

Does anyone know where the "BUT RUST" link was supposed to lead? It seems to be already out of date due to being unversioned, I can't tell whether it's supposed to lead to the middle of the `starts_with` doc comment or not.

Looking at the archive.org captures just before [1] and just after [2] the article was published, it looks like it was meant to be this line of code, now on line 2779 [3]:

            let mid = left + size / 2;

[1] https://web.archive.org/web/20230602210213/https://doc.rust-...

[2] https://web.archive.org/web/20230709221353/https://doc.rust-...

[3] https://doc.rust-lang.org/src/core/slice/mod.rs.html#2779

Re: Fastest branchless binary search

#13
post #8

Interesting that the results don't hold up with a more complicated comp comparison function: > For somewhat realistic scenarios of binary searching with a slower comp() function I’ve thought of searching through ids, phone numbers, accounts and keywords. I’ve thus settled on testing searching 8-byte strings. > ... > In this case std::lower_bound is very slightly but consistently faster than sb_lower_bound. To always…

I can’t find any assertion in the post about the content of any of the input data sets or search keys, other than that they’re “unpredictable”.

I assume purely random, but if these 8-byte strings were anything but pure information, modern branch predictors can sniff their way to better performance than cmov pretty easily.

Re: Fastest branchless binary search

#14
post #2

>gcc has __builtin_expect_with_probability(cond, 0, 0.5) but it does nothing (tested v10). ↩ I wonder what could possibly be the use of this builtin. Branch prediction varies enough between different processors that it seems unlikely that anything useful could be done with a fine-grained probability estimate.

I think it is used for code layout: typically you want to make the most often taken branch as fallthrough as it can be slightly faster even when perfectly predicted. I also tried (and failed) to use expect with probability to generate a cmov, but in retrospect it is obvious: the parameter is not the prediction probability, but the taken/not-taken probability, and even a branch that goes either way with 50% probabilit…

Indeed: A branch taken 50% of the time can be readily predicted 100% of the time if it follows a pattern the CPU can infer.

Why you have to be careful choosing realistic input data if you are microbenchmarking at this level for a real purpose.

Re: Fastest branchless binary search

#15
post #2

>gcc has __builtin_expect_with_probability(cond, 0, 0.5) but it does nothing (tested v10). ↩ I wonder what could possibly be the use of this builtin. Branch prediction varies enough between different processors that it seems unlikely that anything useful could be done with a fine-grained probability estimate.

I think it is used for code layout: typically you want to make the most often taken branch as fallthrough as it can be slightly faster even when perfectly predicted. I also tried (and failed) to use expect with probability to generate a cmov, but in retrospect it is obvious: the parameter is not the prediction probability, but the taken/not-taken probability, and even a branch that goes either way with 50% probabilit…

clang has one:

https://clang.llvm.org/docs/LanguageExtensions.html#builtin-...

Re: Fastest branchless binary search

#16
post #2

>gcc has __builtin_expect_with_probability(cond, 0, 0.5) but it does nothing (tested v10). ↩ I wonder what could possibly be the use of this builtin. Branch prediction varies enough between different processors that it seems unlikely that anything useful could be done with a fine-grained probability estimate.

At least when I last read it a couple years ago, Intel's optimization manual recommends certain code layouts. For example in the absence of better information a conditional jump backwards would be predicted as true (because that's what loops look like) while a jump forward is seen as less likely to happen (error conditions and else branches look like that).

Not sure how consistent this is across architectures, and how useful it still is. But at least it used to be a thing

Re: Fastest branchless binary search

#17
post #8

Interesting that the results don't hold up with a more complicated comp comparison function: > For somewhat realistic scenarios of binary searching with a slower comp() function I’ve thought of searching through ids, phone numbers, accounts and keywords. I’ve thus settled on testing searching 8-byte strings. > ... > In this case std::lower_bound is very slightly but consistently faster than sb_lower_bound. To always…

I believe this happens because branch prediction lets you pipeline multiple simultaneous comparisons, and rewind whenever the branch predictor is wrong (about half the time for truly random data and inputs). The CMOV approach blocks after each comparison function due to the data dependency. On average, you're doing two comparisons at a time with branches, and one with CMOV, so when comparison time is greater than branch prediction penalty, you would expect a crossover to occur.

Re: Fastest branchless binary search

#18

Does anyone know where the "BUT RUST" link was supposed to lead? It seems to be already out of date due to being unversioned, I can't tell whether it's supposed to lead to the middle of the `starts_with` doc comment or not.

Looking at the archive.org captures just before [1] and just after [2] the article was published, it looks like it was meant to be this line of code, now on line 2779 [3]: let mid = left + size / 2; [1] https://web.archive.org/web/20230602210213/https://doc.rust-... [2] https://web.archive.org/web/20230709221353/https://doc.rust-... [3] https://doc.rust-lang.org/src/core/slice/mod.rs.html#2779

Alright, and as a versioned link that'll be: https://github.com/rust-lang/rust/blob/7d8386f05cedf33c7475c...

Thank you!

Re: Fastest branchless binary search

#19

Earlier quoted context omitted.

I think it is used for code layout: typically you want to make the most often taken branch as fallthrough as it can be slightly faster even when perfectly predicted. I also tried (and failed) to use expect with probability to generate a cmov, but in retrospect it is obvious: the parameter is not the prediction probability, but the taken/not-taken probability, and even a branch that goes either way with 50% probabilit…

clang has one: https://clang.llvm.org/docs/LanguageExtensions.html#builtin-...

Which also does not affect cmov conversion. Every time this comes up I bring up this long-standing LLVM bug: https://bugs.llvm.org/show_bug.cgi?id=40027

Re: Fastest branchless binary search

#20

Earlier quoted context omitted.

Looking at the archive.org captures just before [1] and just after [2] the article was published, it looks like it was meant to be this line of code, now on line 2779 [3]: let mid = left + size / 2; [1] https://web.archive.org/web/20230602210213/https://doc.rust-... [2] https://web.archive.org/web/20230709221353/https://doc.rust-... [3] https://doc.rust-lang.org/src/core/slice/mod.rs.html#2779

Alright, and as a versioned link that'll be: https://github.com/rust-lang/rust/blob/7d8386f05cedf33c7475c... Thank you!

You can get versioned link directly into rustdoc:

https://doc.rust-lang.org/1.71.1/src/core/slice/mod.rs.html#...

Post reply on HN