Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

chipsandcheese.com

Post reply on HN
Print view View original

181–190 of 196 posts

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#181

Post by jiggawatts »

JonChesterfield wrote: post #180
jiggawatts wrote: post #164

Earlier quoted context omitted.

We're entering the era of "kilo cores" in the same way computing entered the era of kilobytes in the 1940s. If you consider a tightly-coupled rack of servers with GPUs to be one machine, then we're well into the hundreds of kilocores. I found it entertaining having a debate with someone here on HN who just couldn't grok the concept that it's possible to serve something the size of Wikipedia from a single server. That…

Is that actually true, or only that you can serve it from one server behind a lot of servers named CDN?

It’s definitely true. It won’t be optimal or fast for end users, but it’s entirely doable.

One of the main benefits of a CDN is that it brings the data closer to the users, reducing network latency. Obviously a single server can’t be close to every user globally!

In practice one would always use a CDN for a site like WikiPedia but the point is that it isn’t necessary to handle the traffic.

A single two-socket EPYC server can put out something like 400 Gbps of traffic, which is far more than all but the biggest sites.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#182

Post by topspin »

scheme271 wrote: post #117
topspin wrote: post #99

Earlier quoted context omitted.

> Speculative execution is so valuable for performance that a computer without it is completely unusable. Jim Keller's view aligns with this and goes further. My interpretation of his thinking is that predictors and speculation are the only meaningful features of CPUs today. ISA doesn't matter anymore because the power of modern compilers makes high performance software highly portable and all CPUs end up bottlenecke…

I think a few other things like memory models also matter and affect cpu architecture. E.g. the x86 total store order vs arm64's model. You potentially get to do a few nice optimizations on arm64 vs x86. I'm not sure how much of a difference that makes though.

Memory models are indeed important. That's why Apple extended ARM with total store order capability. A CPU can efficiently implement more than one memory model.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#183

Post by eru »

zaptrem wrote: post #78
fragmede wrote: post #76

Earlier quoted context omitted.

The whole AI trend is in part due to things that are now possible on GPU supercomputers with gigabytes of RAM backed by petabytes of data and at the top speed of GPUs. Some of the algorithms date back to before the ai winter and it's just that we can now do the same thing with a ton more data and faster.

All of the main algorithms do (multi-layer perceptrons and stochastic gradient descent are from the 50s and 60s!). Basically the only thing that changed is we decided to multiply some of the outputs of the multi-layer perceptrons by each other and softmax it (attention) before passing them back into more layers. Almost all of the other stuff is just gravy to make it converge faster (and run faster on modern hardware)…

Is there any indication that people had figured out that simpler activation functions like ReLU are worth bothering?

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#184

Post by zaptrem »

eru wrote: post #183
zaptrem wrote: post #78

Earlier quoted context omitted.

All of the main algorithms do (multi-layer perceptrons and stochastic gradient descent are from the 50s and 60s!). Basically the only thing that changed is we decided to multiply some of the outputs of the multi-layer perceptrons by each other and softmax it (attention) before passing them back into more layers. Almost all of the other stuff is just gravy to make it converge faster (and run faster on modern hardware)…

Is there any indication that people had figured out that simpler activation functions like ReLU are worth bothering?

Oh, forgot about that one. Wikipedia says ReLU was used in NNs in 1969 but not widely until 2011. Idk if anyone has ever trained a transformer with sigmoid activations, but I don’t immediately see why it wouldn’t work?

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#185

Post by Dylan16807 »

paulmd wrote: post #176
Dylan16807 wrote: post #108

Earlier quoted context omitted.

> Regardless though why would it potentially being higher in newer architectures be viewed as a good thing? Because SMT getting faster is a nearly free side-effect. We didn't add extra units to speed up SMT at the cost of single-thread speed. We added extra units to speed up the single thread, and they just happened to speed up SMT even more (at least for the purpose of this theoretical). That's better than speeding…

this doesn't seem to reflect the reality of the way hardware is actually being added to the cores, where Zen5 features a fourth ALU which is only useful to a single thread in the low-single-digits. https://old.reddit.com/r/hardware/comments/1ee7o1d/the_amd_r... this isn't adding more units to speed up single-thread and SMT being a nice "incidental" gain, this is actively targeting wide architectures that have lots of…

What do you think they should add instead of another ALU?

The returns are diminishing, but a single digit bump is still a pretty good when your goal is faster single threads.

Adding nothing to save space is obviously not the answer, because that leads to having more slower cores.

Also the topic of the post, the 2-ahead branch predictor, exists specifically to get more ALUs running in parallel!

> apple silicon, where the focus is running super deep speculation

And to make that speculation profitable, the M1 cores have 6 ALUs among other wideness.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#186

Post by yogrish »

When I began my career as a DSP optimization engineer, I worked on ZSP processors. I had a particular passion for branch predictions. By analyzing data, we would code and recode branch instructions to minimize mispredictions. I loved that super- scalar architecture of ZSP and its Branch prediction feature was amazing, apart from its multi Instruciton execution in a cycle.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#187

Post by protomolecule »

pcwalton wrote: post #177
protomolecule wrote: post #105

Earlier quoted context omitted.

"with some new ideas from C++ of the time" Could you elaborate on that?

Move semantics and rvalue references. It was clear that within C++11 there was a substructural type system struggling to get out.

I was going to say that Rust's destructive move is quite different from C++'s move semantics, but official docs disagree: "C++: references, RAII, smart pointers, move semantics, monomorphization, memory model" [0]

[0] https://doc.rust-lang.org/reference/influences.html

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#188

Post by account42 »

Dylan16807 wrote: post #146
bayindirh wrote: post #93

Earlier quoted context omitted.

Serving web pages is cheap. You’ll probably hit network I/O limits before you saturate the cores. I wonder what about its HPC performance. I think cooling this won’t be big problem, but might be wet one, requiring DLC after a certain point.

> Serving web pages is cheap. You’ll probably hit network I/O limits before you saturate the cores. It's hard to be network I/O bound when serving web pages. Netflix struggles to be network I/O bound when serving video , which is so much bigger and uses so much less processing. Epyc started off with 32 cores on PCIe 3, and quickly moved to 64 cores on PCIe 4. When we hit 256 cores it's probably going to have PCIe 6,…

Video being bigger also means it does not fit in lower levels of the cache hierarchy. For a normal website 99% of views should be served entirely from RAM.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#189

Post by nomel »

immibis wrote: post #170
nomel wrote: post #26

Earlier quoted context omitted.

With the SMT core count having to be one less.

No.

Could you expand on that a bit? How could the "same circuit" be used for twice as many things, for free. If it's currently being used as an SMT core, how could it also support the second branch prediction, for both SMT cores?

I say the SMT core would be reduced by one since, rather than being used as an SMT core, it would be used as the second branch prediction thing.

Re: Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks

#190

Post by eru »

zaptrem wrote: post #184
eru wrote: post #183

Earlier quoted context omitted.

Is there any indication that people had figured out that simpler activation functions like ReLU are worth bothering?

Oh, forgot about that one. Wikipedia says ReLU was used in NNs in 1969 but not widely until 2011. Idk if anyone has ever trained a transformer with sigmoid activations, but I don’t immediately see why it wouldn’t work?

I remember some experiments of using modern day training and data on some old style networks, eg with sigmoid activation.

That worked eventually and worked quite well, but took way more compute and training data that anyone back in the olden days would have thought feasible.

The two main problems with sigmoid activation compared to ReLU are: (a) harder to compute (both the value itself and the gradient), and (b) vanishing gradients, especially in deeper networks.

Post reply on HN