Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dynamic Branch Prediction with Perceptrons (2000) [pdf]

cs.cmu.edu

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#11

Post by kordlessagain »

pfdietz wrote: post #9
bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

But what's the impact of those 5% misses?

bot> Querying GPT...

bot> A 5% miss rate in branch prediction leads to a 14.7% improvement in misprediction rates on a trace of SPEC2000 benchmarks compared to the gshare predictor. The use of machine learning-based predictors has the potential to improve these results further.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#12

Post by jackmott42 »

peterfirefly wrote: post #5
jacknews wrote: post #3

It's an interesting idea. Did this go anywhere? Is it worth revisiting in 2023?

That is a* very, very important paper. The answer is yes. (The guy who posted the link is Dan Luu who used to work for a company that made x86 CPUs. His blog is worth looking at. He also used to work for Twitter -- but his predictions about the Doom and Disaster after Musk's takeover hasn't panned out at all. Good at some things (CPU architecture and blog posts). Not so good at Elon Musk predictions ;) ) --- Edit: pu…

[flagged]

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#14

Post by p_l »

bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

AMD been using perceptrons in their branch prediction logic since I think K6-2.

Some updates to it were big point of "what we did in Zen" presentations when first Ryzen and EPYC CPUs landed.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#15

Post by titzer »

bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

> relatively simple state machines

To be clear, all hardware branch predictors are "relatively simple state machines"; they need storage in the branch prediction tables which must super-fast to access, which means they can only store a few (sometimes dozens, but certainly not hundreds) bits per branch to reach the access latency goal. With input to the predictor encoded as binary, the weights quantized and small and encoded into binary, and the history small, even perceptrons are "relatively simple state machines". After all, their implementation is just going to become some combinatorial logic in the end.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#16

Post by gpderetta »

pfdietz wrote: post #9
bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

But what's the impact of those 5% misses?

Huge. I think 95% was table stakes 30 years ago. IIRC the average rate of a good predictor these days is 99.5%.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#17

Post by isotypic »

thesz wrote: post #8
albertzeyer wrote: post #7

Earlier quoted context omitted.

But is it really much more complicated? It looks very simple to me. Maybe actually even simpler than such state machine?

There are several multipliers and adders (possible, floating point ones), a register file for perceptron weights and perceptron selection logic. The scheme also need to keep information somewhere for training - the decision and backpropagation stimulus are separeted in time. So the circuitry is complicated despite superficial simplicity of the model.

The version in the paper only is ever multiplying by 1 or -1, so you do not need a full multiplier circuit. The weights are also stored as signed integers, not floats, so no complicated floating point circuity. I am not sure what current state of the art is, but considering the cost of multipliers/floating point circuity I would be surprised if it changed to those if signed integers work.

All branch predictors need some way of storing their state and selection logic, and the way a perceptron branch predictor stores its data is just a big table indexed by some hash of the program counter of the branch, which is pretty standard for branch predictors. Also, all branch predictors have a sort of "backpropogation" in that pipelined processors produce the actual result of the branch (possibly many) cycles later, so this also is not as much of a factor. Since the training is a function of the weights you do not need to store extra data beyond the threshold, but that is already being computed as the prediction anyways.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#18

Post by mirker »

bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

One thing to point out is that the threshold of predictor complexity is dependent on the execution pipeline. A very speculative and deep architecture has a bigger need for better predictors, since it has a massive penalty when there is a misprediction.

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#19

Post by delta_p_delta_x »

Modern CPUs use TAGE-like[1] and Perceptron-based branch predictors, achieving a 99.7% prediction accuracy. Zen 2 in particular uses TAGE[2].

I recall implementing one in software in a computer architectures class; it was pretty gnarly but the prediction accuracy (and therefore performance) compared to a simple two-bit saturating counter is immense.

[1]: https://doi.org/10.1145/3226098 [2]: https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Br...

Re: Dynamic Branch Prediction with Perceptrons (2000) [pdf]

#20

Post by artisanspam »

bjourne wrote: post #6

Modern branch predictors are based on relatively simple state machines and already have >95% accuracy. Thus, even if machine learning-based predictors can sometimes beat them, it is not clear that they can beat them enough for the much more complicated circuitry they need to be worth it.

As someone who works on design verification for modern CPUs, this is not true. Most modern CPUs use some form of TAGE branch prediction, if not something more complex.
Post reply on HN