Live data from Hacker News

How many branches can your CPU predict?

lemire.me

51–60 of 66 posts

Re: How many branches can your CPU predict?

#51

By only testing one static branch, it is possible that the performance of the Intel Emerald Rapids predictor is not representative of a more realistic workload. If path information is used to index the predictor in addition to global (taken/not taken) branch history without xoring with the global history (or fulling mingling these different data) or if the branch address is similarly not fully scrambled with the glob…

I use a similar conditional write paradigm on the GPU and it's usually easiest to do an unconditional write and update the address using a branchless conditional, assuming you are using a system with strict write ordering. Usually the unnecessary writes won't make it out of L1 cache.

Re: How many branches can your CPU predict?

#52
post #17
post #6

Hmm, that's interesting. The code as written only has one branch, the if statement (well, two, the while loop exit clause as well). My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. But that's clearly not right, because apparently the specific data it's branch…

Your mental model is close. Predictors generally work by having some sort of table of predictions and indexing into that table (usually using some sort of hashing) to obtain the predictions. The simplest thing to do is use the address of the branch instruction as the index into the table. That way, each branch instruction maps onto a (not necessarily unique) entry in the table. Those entries will usually be a two-bit…

Yeah, the "two-bit saturating counter" thing is pretty much exactly how I thought it worked (which would be terrible for the example in the article), but I had no idea about the fact that it also kept track of the branch history thing, and how that maps to different branch predictor entries. Thanks for the link, that's really fascinating!

Re: How many branches can your CPU predict?

#53

Earlier quoted context omitted.

It seems like that would struggle with detecting how many layers of branching to pay attention to. Imagine the two nested loops surrounded by a randomized one. Wouldn't that implementation keep hitting patterns it hadn't seen before? Obviously that must be a solved problem; I'd be curious to know what the solution is.

might be but what real code does that ?

[deleted]

Re: How many branches can your CPU predict?

#54
post #6

Hmm, that's interesting. The code as written only has one branch, the if statement (well, two, the while loop exit clause as well). My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. But that's clearly not right, because apparently the specific data it's branch…

> My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. I always figured the algorithm was much simpler, it would just use the same branch as last execution — should work fairly well. Didn’t realize it used the input value as well, which to me makes no sense — the…

Agreed. I wonder if this silicon is designed for this benchmark and if not how useful is it with real code.

I would be surprised if this silicon area could not be better utilized for something else

Re: How many branches can your CPU predict?

#55
post #17
post #6

Hmm, that's interesting. The code as written only has one branch, the if statement (well, two, the while loop exit clause as well). My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. But that's clearly not right, because apparently the specific data it's branch…

Your mental model is close. Predictors generally work by having some sort of table of predictions and indexing into that table (usually using some sort of hashing) to obtain the predictions. The simplest thing to do is use the address of the branch instruction as the index into the table. That way, each branch instruction maps onto a (not necessarily unique) entry in the table. Those entries will usually be a two-bit…

It even gets much more sophisticated than that. Even the first Ryzen had something perceptron-based (yes, neuronal network!) and there are several predictors + logic to pick the best one for a branch.

Re: How many branches can your CPU predict?

#57
post #56

I'm wondering why my submission, made 22 hours ago, is marked as a duplicate, but this submission, made just 12 hours ago, isn't. https://news.ycombinator.com/item?id=47438490

I'm also wondering why the same URL for both submissions isn't recognized as the same, and why the new submission was allowed.

Re: How many branches can your CPU predict?

#58
post #6

Hmm, that's interesting. The code as written only has one branch, the if statement (well, two, the while loop exit clause as well). My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. But that's clearly not right, because apparently the specific data it's branch…

> My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. I always figured the algorithm was much simpler, it would just use the same branch as last execution — should work fairly well. Didn’t realize it used the input value as well, which to me makes no sense — the…

> I always figured the algorithm was much simpler, it would just use the same branch as last execution — should work fairly well.

Sure, that would work significantly better than no predictor at all. But you'd agree that a better predictor would work better, right? The missing detail might be how expensive mispredicted branches are compared to other costs. If you can go from 50% accuracy to 90% accuracy, it wouldn't be surprising to more than double your performance.

> Didn’t realize it used the input value as well, which to me makes no sense — the whole point is to avoid having to inspect the value.

It doesn't, and can't for the reasons you hint at. The reason branch prediction is necessary is that the value often isn't available yet when the branch is taken. Was there something in the article that implied the opposite?

--

I wonder if Daniel's tricksy approach using a random number generator to simulate a complex pattern is misleading people here.

One of the main benefits of branch prediction is predicting the end of a loop, particularly, a loop within a loop. In assembly, a loop is just a comparison at the end and a branch back to the beginning. Assume you had a loop that always executes 8 times, or some other small fixed value. Also assume there is some reason you can't unroll that loop, and that loop is inside another loop that executes millions of times. It's a real boost to performance if you can consistently predict the end of the inner loop.

If you predicted just on the last time the loop closing branch was taken, you'd always miss the ending. But if you can remember a pattern that is longer than 8, you can always get it right. This is obviously valuable. The bigger question is how much more valuable it is to predict a loop (where "loop" might actually be a complex execution pattern across multiple branches) that is thousands long rather than just 8. But quantifying how long this pattern can be on different processors is part of the groundwork for analyzing this.

Re: How many branches can your CPU predict?

#59
post #6

Hmm, that's interesting. The code as written only has one branch, the if statement (well, two, the while loop exit clause as well). My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. But that's clearly not right, because apparently the specific data it's branch…

> My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. I always figured the algorithm was much simpler, it would just use the same branch as last execution — should work fairly well. Didn’t realize it used the input value as well, which to me makes no sense — the…

It does not and cannot use the input value. The branch predictor runs like a dozen cycles before execution, it generally cannot possibly see values in registers. It also runs like 3-4 cycles before decode, so it cannot even use any bits from the instruction that is being executed.⁰ Branch predictors strictly use branch history, but this is more complex than just looking at the probability of branching on a single branch, there are things like tables that maintain all branches over the past tens of thousands of cycles, and try to cover common patterns.

0: This is why the first prediction is always "don't branch", because the first time executing code the predictor has literally no information at all. Every now and then people ask for hint bits on branches, but, er, how are you planning to do that when the instruction with the branch hasn't arrived from L1 when the prediction is due?

Re: How many branches can your CPU predict?

#60

Earlier quoted context omitted.

> My mental model of the branch predictor was that for each branch, the CPU maintained some internal state like "probably taken/not taken" or "indeterminate", and it "learned" by executing the branch many times. I always figured the algorithm was much simpler, it would just use the same branch as last execution — should work fairly well. Didn’t realize it used the input value as well, which to me makes no sense — the…

Agreed. I wonder if this silicon is designed for this benchmark and if not how useful is it with real code. I would be surprised if this silicon area could not be better utilized for something else

No, branch predictors are really important and even small improvements in them are extremely valuable on real loads. Improving branch prediction is both a power and a performance optimization.
Post reply on HN