Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

131–140 of 186 posts

Re: What every developer should know about GPU computing

#131

Post by PeterisP »

tormeh wrote: post #121
jacquesm wrote: post #105

Earlier quoted context omitted.

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

If AMD really wished ML researchers to use their hardware, they could have put in the effort to make OpenCL or Vulkan competitive with CUDA with respect to ease of developing ML systems. nVidia did put in that software effort, AMD didn't so now they both reap the consequences.

Re: What every developer should know about GPU computing

#132

Post by PeterisP »

chpatrick wrote: post #112
tester756 wrote: post #104

Earlier quoted context omitted.

"Going through a formal CS program will give you an intimate understanding of CPUs and 101 other hilarious jokes you can tell yourself!"

Depends on the university! In my school to pass the computer architecture course you had to read and present a recent paper on CPU design.

Which most of these students will have entirely forgotten a few years after graduation - use it or lose it.

Re: What every developer should know about GPU computing

#133

Post by guidedlight »

One thing I don’t understand is how the architecture of Apple Silicon is different from NVidia’s.

Looking at this quote:

> the Nvidia H100 GPU has 132 SMs with 64 cores per SM, totalling a whopping 8448 cores.

8448 cores sure sounds impressive. But the Apple M2 Ultra only has 76 cores?!

How can the NVidia H100 GPU have over 110x more cores? Clearly it doesn’t have 110x more performance over the M2 Ultra, so what is going on here?

Re: What every developer should know about GPU computing

#134

Post by p1esk »

behnamoh wrote: post #125
tormeh wrote: post #121

Earlier quoted context omitted.

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

Those ml researchers wanted to get hired…

Those ML researchers (myself included) wanted to get the job done.

Re: What every developer should know about GPU computing

#135

Post by p1esk »

anonporridge wrote: post #49
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

Definitely not true about most programmers, but maybe the author meant CS educated engineers. Going through a formal CS program will give you an intimate understanding of CPUs, especially when compared to the very light coverage of GPUs.

Going through a formal CS program will give you an intimate understanding of CPUs

Please tell me you forgot the /s.

I have a PhD in computer engineering from a top-20 school in US. Took a bunch of grad level classes, passed the quals (my specialty was ML accelerators).

I do NOT have an “intimate understanding of CPUs”. I probably know a little bit more about CPUs than an average programmer. Which is very little.

Modern CPUs are extremely complex. Almost as much of impenetrable black boxes as modern neural networks.

Re: What every developer should know about GPU computing

#136

Post by kevingadd »

guidedlight wrote: post #133

One thing I don’t understand is how the architecture of Apple Silicon is different from NVidia’s. Looking at this quote: > the Nvidia H100 GPU has 132 SMs with 64 cores per SM, totalling a whopping 8448 cores. 8448 cores sure sounds impressive. But the Apple M2 Ultra only has 76 cores?! How can the NVidia H100 GPU have over 110x more cores? Clearly it doesn’t have 110x more performance over the M2 Ultra, so what is g…

NVIDIA's SMs are most comparable to the 'CUs' on AMD GPUs or cores on Apple GPUs, generally speaking. The "cores" are subsets of the SM that perform individual operations, IIRC.

See this diagram from an nvidia blog post: https://developer-blogs.nvidia.com/wp-content/uploads/2021/g...

( https://developer.nvidia.com/blog/nvidia-ampere-architecture... )

Re: What every developer should know about GPU computing

#137

Post by kevingadd »

rakkhi wrote: post #124

Imagine with NVIDIA banned in china, how well the Chinese local companies will do in GPU's for AI: https://x.com/BeijingDai/status/1715861773495279743?s=20

It's going to take them a while to catch up to CUDA, though. Even with stolen IP it's going to be tough to make 1:1 drop-in replacements. The amount of existing investment into AMD and NVIDIA architectures is huge, as evidenced by how bad the Moore Threads GPUs are - it simply isn't easy to enter the market as a serious competitor. Even Intel is struggling.

Re: What every developer should know about GPU computing

#138

Post by jandrewrogers »

anonporridge wrote: post #49
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

Definitely not true about most programmers, but maybe the author meant CS educated engineers. Going through a formal CS program will give you an intimate understanding of CPUs, especially when compared to the very light coverage of GPUs.

For anyone that needs to know how a CPU works for performance engineering similar to a GPU, the details of the microarchitecture matter a lot even within the same ISA. I am not aware of any formal CS program that teaches anyone the nuanced internals of various microarchitecture designs. Everyone I know with this knowledge appears to be self-taught regardless of where they went to school.

I think the descriptor “intimate” is overstating the case if you don’t know how to optimize code on different implementations of the same ISA. Most formal CS programs give you a generic understanding of CPUs, more like a survey course, not enough information to do serious optimization.

Re: What every developer should know about GPU computing

#139

Post by subharmonicon »

guidedlight wrote: post #133

One thing I don’t understand is how the architecture of Apple Silicon is different from NVidia’s. Looking at this quote: > the Nvidia H100 GPU has 132 SMs with 64 cores per SM, totalling a whopping 8448 cores. 8448 cores sure sounds impressive. But the Apple M2 Ultra only has 76 cores?! How can the NVidia H100 GPU have over 110x more cores? Clearly it doesn’t have 110x more performance over the M2 Ultra, so what is g…

NVIDIA is intentionally being obtuse and frankly dishonest calling what’s effectively a vector lane a “core” and similarly using “thread” in “SIMT” to mean the execution of one of those vector lanes.

Yes, their architecture is different from many in that they support a separate program counter per lane (which is why they feel justified in calling this a “thread”), but ultimately it’s the rate and throughput of ALUs that matter.

Re: What every developer should know about GPU computing

#140

Post by HalfCrimp »

jokoon wrote: post #63

I wish it was easier to program a GPU... I've already refrained myself to learn vulkan because it scares me, but similarly, opengl and cuda are a bit mysterious to me, and I don't really know how I could take advantage of it, since most computing tasks cannot be made parallel. I've read there are data structures that are somehow able to take advantage of a GPU as an alternative to the CPU (for example a database runn…

If you are comfortable with C++ already then look at Thrust. It's nvidia's analogue to the standard library arrived at GPU computing.

Writing and launching raw cuda kernels is too low level for me, but writing with Thrust makes it feel pretty similar to writing regular C++ code. You still need to deal with moving data from host to device and back, but that's as simple as assigning a `thrust::device_vector` to a `thrust::host_vector`

Post reply on HN