Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Think About GPUs

jax-ml.github.io

Post reply on HN
Print view View original

121–127 of 127 posts

Re: How to Think About GPUs

#121

Post by GregarianChild »

adrian_b wrote: post #15
achierius wrote: post #13

Earlier quoted context omitted.

That's not true. SIMT notably allows for divergence and reconvergence, whereby single threads actually end up executing different work for a time, while in SIMD you have to always be in sync.

"Divergence" is supported by any SIMD processor, but with various amounts of overhead depending on the architecture. "Divergence" means that every "divergent" SIMD instruction is executed at least twice, with different masks, so that it is actually executed only on a subset of the lanes (i.e. CUDA "threads"). SIMT is a programming model, not a hardware implementation. NVIDIA has never explained exactly how the execut…

This paper

https://arxiv.org/abs/2407.02944

ventures some guesses how Nvidia does this, and runs experiments to confirm them.

Re: How to Think About GPUs

#122

Post by almostgotcaught »

dang wrote: post #117
almostgotcaught wrote: post #109

Earlier quoted context omitted.

then please enlighten me: what does the sentence mean?

Can you please stop posting in the cross-examining, flamewar style? It's not what this site is for. We want people to learn from each other here. https://news.ycombinator.com/newsguidelines.html

what exactly is the acceptable style of discourse here then? purely exultant tropes and clickbait i guess.

Re: How to Think About GPUs

#123

Post by HarHarVeryFunny »

cwmoore wrote: post #111
HarHarVeryFunny wrote: post #102

Earlier quoted context omitted.

Meaning what? Something less flexible? Less CUDA cores and more Tensor Cores? The majority of NVidia's profits (almost 90%) do come from data center, most of which is going to be neural net acceleration, and I'd have to assume that they have optimized their data center products to maximize performance for typical customer workloads. I'm sure that Microsoft would provide feedback to Nvidia if they felt changes were ne…

> most of which is going to be neural net acceleration is it?

I've got to assume so, since data center revenue growth seems to have grown in sync with recent growth in AI adoption. CUDA has been around for a long time, so it would seem highly coincidental if non-AI CUDA usage was only just now surging at same time as AI usage is taking off, and new data center build announcements seem to invariably be linked to AI.

Re: How to Think About GPUs

#124

Post by dang »

almostgotcaught wrote: post #122
dang wrote: post #117

Earlier quoted context omitted.

Can you please stop posting in the cross-examining, flamewar style? It's not what this site is for. We want people to learn from each other here. https://news.ycombinator.com/newsguidelines.html

what exactly is the acceptable style of discourse here then? purely exultant tropes and clickbait i guess.

Does https://news.ycombinator.com/newsguidelines.html not answer that?

There are plenty of acceptable styles. The guidelines don't insist on only one style.

Re: How to Think About GPUs

#125

Post by jacobaustin123 »

hackrmn wrote: post #106
jacobaustin123 wrote: post #76

Earlier quoted context omitted.

Shamelessly responding as the author. I (mostly) agree with you here. > please be surgically precise with your terms There's always a tension between precision in every explanation and the "moral" truth. I can say "a SIMD (Single Instruction Multiple Data) vector unit like the TPU VPU with 32 ALUs (SIMD lanes) which NVIDIA calls CUDA Cores", which starts to get unwieldy and even then leaves terms like vector units un…

I appreciate your response. I made a point of not revising my comment after posting it and finding in a subsequent parable the following, quoting: > Each SM is broken up into 4 identical quadrants, which NVIDIA calls SM subpartitions, each containing a Tensor Core, 16k 32-bit registers, and a SIMD/SIMT vector arithmetic unit called a Warp Scheduler, whose lanes (ALUs) NVIDIA calls CUDA Cores. And right after: > CUDA…

Thank you for taking the time to write this reply. Agree with "in the previous part of this series" comment. I'll try to find a way to highlight this more.

What I'd like to add to this page is some sort of highly clear glossary that defines all the terms at the top (but in some kind of collapsable fashion) so I can define everything with full clarity without disrupting the flow. I'll play with the HTML and see what I can do.

Re: How to Think About GPUs

#126

Post by nickysielicki »

aschleck wrote: post #8
nickysielicki wrote: post #7

The calculation under “Quiz 2: GPU nodes“ is incorrect, to the best of my knowledge. There aren’t enough ports for each GPU and/or for each switch (less the crossbar connections) to fully realize the 450GB/s that’s theoretically possible, which is why 3.2TB/s of internode bandwidth is what’s offered on all of the major cloud providers and the reference systems. If it was 3.6TB/s, this would produce internode bottlene…

It's been a while since I thought about this but isn't the reason providers advertise only 3.2tbps because that's the limit of a single node's connection to the IB network? DGX is spec'ed to pair each H100 with a Connect-X 7 NIC and those cap out at 400gbps. 8 gpus * 400gbps / gpu = 3.2tbps. Quiz 2 is confusingly worded but is, iiuc, referring to intranode GPU connections rather than internode networking.

Try running a single node all to all with sharp disabled. I don’t believe you’ll see 450GB/s.
Post reply on HN