Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

141–150 of 186 posts

Re: What every developer should know about GPU computing

#141

Post by tormeh »

PeterisP wrote: post #131
tormeh wrote: post #121

Earlier quoted context omitted.

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

If AMD really wished ML researchers to use their hardware, they could have put in the effort to make OpenCL or Vulkan competitive with CUDA with respect to ease of developing ML systems. nVidia did put in that software effort, AMD didn't so now they both reap the consequences.

No argument there. AMD really shat the bed. But if you go with a single supplier, it’s pretty weird to make the pricing agreement “we’ll pay whatever you charge”.

Re: What every developer should know about GPU computing

#142

Post by freeone3000 »

subharmonicon wrote: post #139
guidedlight wrote: post #133

One thing I don’t understand is how the architecture of Apple Silicon is different from NVidia’s. Looking at this quote: > the Nvidia H100 GPU has 132 SMs with 64 cores per SM, totalling a whopping 8448 cores. 8448 cores sure sounds impressive. But the Apple M2 Ultra only has 76 cores?! How can the NVidia H100 GPU have over 110x more cores? Clearly it doesn’t have 110x more performance over the M2 Ultra, so what is g…

NVIDIA is intentionally being obtuse and frankly dishonest calling what’s effectively a vector lane a “core” and similarly using “thread” in “SIMT” to mean the execution of one of those vector lanes. Yes, their architecture is different from many in that they support a separate program counter per lane (which is why they feel justified in calling this a “thread”), but ultimately it’s the rate and throughput of ALUs t…

It’s not even a seperate PC per lane, you only get that per block — lane level execution goes to an execution mask lut per insn. Branchy code that’s not branchy uniformly in the block executes a lot of noops.

Re: What every developer should know about GPU computing

#144

Post by markhahn »

tormeh wrote: post #121
jacquesm wrote: post #105

Earlier quoted context omitted.

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

huh? don't ML researchers use pytorch and tensorflow?

Re: What every developer should know about GPU computing

#145

Post by [deleted] »

twixfel wrote: post #114
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

I don't understand why every other submission on the internet has to have at least one "stopped reading at X" comment relating to it. It adds absolutely nothing.

[deleted]

Re: What every developer should know about GPU computing

#146

Post by mr_toad »

guidedlight wrote: post #133

One thing I don’t understand is how the architecture of Apple Silicon is different from NVidia’s. Looking at this quote: > the Nvidia H100 GPU has 132 SMs with 64 cores per SM, totalling a whopping 8448 cores. 8448 cores sure sounds impressive. But the Apple M2 Ultra only has 76 cores?! How can the NVidia H100 GPU have over 110x more cores? Clearly it doesn’t have 110x more performance over the M2 Ultra, so what is g…

For one thing you can use the H100 to heat a room - it uses more than 10x the power of an M2 Ultra.

Re: What every developer should know about GPU computing

#147

Post by permo-w »

twixfel wrote: post #114
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

I don't understand why every other submission on the internet has to have at least one "stopped reading at X" comment relating to it. It adds absolutely nothing.

I regret the way I phrased it. it's snarky and not in the spirit of curiosity, but I do think it's worth discussing that the author thinks that most programmers know how a CPU works

Re: What every developer should know about GPU computing

#148

Post by 01100011 »

nine_k wrote: post #95
01100011 wrote: post #45

> Copying Data from Host to Device Surprised there's no mention of async copies here. If you want to get the most out of the GPU, you don't want it idle when copying data between the host and the GPU. Many frameworks provide for a mechanism to schedule async copies which can execute along side async work submission. The post is sort of GPU 101 but there's a whole world of tricks and techniques beyond that once you st…

Since you likely use 64-bit (double) floats, not every GPU would help much, especially compared to a beefy CPU. But if you use a GPU with a large number of FP64 units, it may speed things up a lot. These are generally not gaming GPUs, but if you have a 4060 sitting around anyway, it has about 300 GFLOPS FP64 performance, likely more than your CPU. Modern CPUs are mighty in this regard though, able to issue many FP64…

Did you reply to the wrong comment?

Re: What every developer should know about GPU computing

#149

Post by tantalor »

chpatrick wrote: post #112
tester756 wrote: post #104

Earlier quoted context omitted.

"Going through a formal CS program will give you an intimate understanding of CPUs and 101 other hilarious jokes you can tell yourself!"

Depends on the university! In my school to pass the computer architecture course you had to read and present a recent paper on CPU design.

That's a lot to ask of an undergrad

Re: What every developer should know about GPU computing

#150

Post by nanidin »

markhahn wrote: post #144
tormeh wrote: post #121

Earlier quoted context omitted.

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

huh? don't ML researchers use pytorch and tensorflow?

I’m guessing we’re talking about the pioneers here.
Post reply on HN