Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

151–160 of 186 posts

Re: What every developer should know about GPU computing

#151

Post by sakras »

freeone3000 wrote: post #142
subharmonicon wrote: post #139

Earlier quoted context omitted.

NVIDIA is intentionally being obtuse and frankly dishonest calling what’s effectively a vector lane a “core” and similarly using “thread” in “SIMT” to mean the execution of one of those vector lanes. Yes, their architecture is different from many in that they support a separate program counter per lane (which is why they feel justified in calling this a “thread”), but ultimately it’s the rate and throughput of ALUs t…

It’s not even a seperate PC per lane, you only get that per block — lane level execution goes to an execution mask lut per insn. Branchy code that’s not branchy uniformly in the block executes a lot of noops.

As of Volta, they have independent PCs with a warp optimizer that dynamically groups threads with the same program counter, so branches aren’t nearly as bad as they used to be.

Re: What every developer should know about GPU computing

#152

Post by rnrn »

fancyfredbot wrote: post #97

This article is pretty good but, looking at comments so far, nobody seems to have made the obvious/predictable point that it's terribly Nvidia specific. That would be understandable perhaps a few years ago. But the era where there was no reasonable alternative is over. Sycl is a good language which performs well across multiple hardware vendors. Sapphire rapids is very good hardware. AMD's MI300 looks amazing. Nvidia…

Sapphire Rapids is a CPU.

AMD's primary focus for a GPU software ecosystem these days seems to be implementing CUDA with s/cuda/hip

Re: What every developer should know about GPU computing

#153

Post by latency-guy2 »

tantalor wrote: post #149
chpatrick wrote: post #112

Earlier quoted context omitted.

Depends on the university! In my school to pass the computer architecture course you had to read and present a recent paper on CPU design.

That's a lot to ask of an undergrad

Maybe a modern one for a degree mill that cares not for breadth or depth. It really shouldn't be otherwise.

I'm very confident in people having the ability to read by the time they are in college. And considering that summarizing a research paper doesn't even have to be perfect, plus very little need to scrutinize the experiment itself, undergrads should be able to do that.

Otherwise they don't belong in college.

Re: What every developer should know about GPU computing

#154

Post by rnrn »

HalfCrimp wrote: post #140
jokoon wrote: post #63

I wish it was easier to program a GPU... I've already refrained myself to learn vulkan because it scares me, but similarly, opengl and cuda are a bit mysterious to me, and I don't really know how I could take advantage of it, since most computing tasks cannot be made parallel. I've read there are data structures that are somehow able to take advantage of a GPU as an alternative to the CPU (for example a database runn…

If you are comfortable with C++ already then look at Thrust. It's nvidia's analogue to the standard library arrived at GPU computing. Writing and launching raw cuda kernels is too low level for me, but writing with Thrust makes it feel pretty similar to writing regular C++ code. You still need to deal with moving data from host to device and back, but that's as simple as assigning a `thrust::device_vector` to a `thru…

or look at using standard C++ with an implementation that uses the GPU: https://docs.nvidia.com/hpc-sdk/compilers/c++-parallel-algor...

Re: What every developer should know about GPU computing

#155

Post by nomel »

PeterisP wrote: post #131
tormeh wrote: post #121

Earlier quoted context omitted.

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

If AMD really wished ML researchers to use their hardware, they could have put in the effort to make OpenCL or Vulkan competitive with CUDA with respect to ease of developing ML systems. nVidia did put in that software effort, AMD didn't so now they both reap the consequences.

Related, I’m putting some money into AMD. They can only improve, at this point.

Re: What every developer should know about GPU computing

#156

Post by subharmonicon »

sakras wrote: post #151
freeone3000 wrote: post #142

Earlier quoted context omitted.

It’s not even a seperate PC per lane, you only get that per block — lane level execution goes to an execution mask lut per insn. Branchy code that’s not branchy uniformly in the block executes a lot of noops.

As of Volta, they have independent PCs with a warp optimizer that dynamically groups threads with the same program counter, so branches aren’t nearly as bad as they used to be.

Can you cite a reference explaining this ability to re-form new warps from existing threads?

I ask because I’ve seen posts from NVIDIA support saying that divergence is still very expensive and I’ve also seen benchmarks that force divergence in each warp by evenly splitting the warp, and the benchmarks result in 2x runtime when that happens vs. when the control-flow is dynamically uniform.

One thing to keep in mind is that even if you were to dynamically reform-warps, there’s still a potential expense because you then lose the advantage of doing things like accessing adjacent elements of data in adjacent threads. You’re bound to now have more bank conflicts, fewer memory accesses being coalesced, etc. Perhaps they do actually do this warp re-formation, but that itself does have this additional cost.

Re: What every developer should know about GPU computing

#157

Post by gmadsen »

p1esk wrote: post #135
anonporridge wrote: post #49

Earlier quoted context omitted.

Definitely not true about most programmers, but maybe the author meant CS educated engineers. Going through a formal CS program will give you an intimate understanding of CPUs, especially when compared to the very light coverage of GPUs.

Going through a formal CS program will give you an intimate understanding of CPUs Please tell me you forgot the /s. I have a PhD in computer engineering from a top-20 school in US. Took a bunch of grad level classes, passed the quals (my specialty was ML accelerators). I do NOT have an “intimate understanding of CPUs”. I probably know a little bit more about CPUs than an average programmer. Which is very little. Mode…

I think the intention of the phrase, is that most cs programs include a systems course. you will learn von neumann arch, with cpu including {registers, clock, ALUs, etc}. Obviously modern CPUs have a lot more complexity, but even the basic coverage of CPUs is more coverage than GPUS.

Re: What every developer should know about GPU computing

#158

Post by bartwr »

tormeh wrote: post #121
jacquesm wrote: post #105

Earlier quoted context omitted.

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

You have clearly not programmed in any of those. :)

OpenCL was almost unusable due to how difficult it was to set up, plus limited support. Vulcan is terribly ugly and verbose and provides no benefits for pure compute workloads. Writing a "hello word" is a few thousands lines of code, debugging impossible.

Working with CUDA is pure pleasure - immediate setup, super easy CPU/GPU interop and code sharing, all the modern (at the time!) C++ features, super pleasant debugging with stepping, profilers, everything. :)

I totally wish others invested as much into tools as NVIDIA did - but they didn't and made the whole experience miserable. :( If your velocity is 1/10 because of terrible tooling, you need to expect some insane benefits to pick it.

Re: What every developer should know about GPU computing

#159

Post by pca006132 »

jandrewrogers wrote: post #138
anonporridge wrote: post #49

Earlier quoted context omitted.

Definitely not true about most programmers, but maybe the author meant CS educated engineers. Going through a formal CS program will give you an intimate understanding of CPUs, especially when compared to the very light coverage of GPUs.

For anyone that needs to know how a CPU works for performance engineering similar to a GPU, the details of the microarchitecture matter a lot even within the same ISA. I am not aware of any formal CS program that teaches anyone the nuanced internals of various microarchitecture designs. Everyone I know with this knowledge appears to be self-taught regardless of where they went to school. I think the descriptor “intim…

I am really curious about what kind of optimizations are enabled when you know the microarchitectural design of various CPUs, but without writing assembly by hand. I only know the basics such as optimizing data structure for better cache locality, add some fast paths, manual unrolling, but have no idea about how to work around things like pipeline stalls. It would be really helpful if you can point to some materials about this!

Re: What every developer should know about GPU computing

#160

Post by dang »

Someone emailed to complain about this:

https://twitter.com/abhi9u/status/1715753871564476597

That is against HN's rules. In fact, it's the one thing that's important enough to be in both the site guidelines and FAQ. HN users feel extremely strongly about this.

Q: Can I ask people to upvote my submission?

A: No. Users should vote for a story because they personally find it intellectually interesting, not because someone has content to promote. We penalize or ban submissions, accounts, and sites that break this rule, so please don't.

https://news.ycombinator.com/newsfaq.html

Don't solicit upvotes, comments, or submissions. Users should vote and comment when they run across something they personally find interesting—not for promotion.

https://news.ycombinator.com/newsguidelines.html

Post reply on HN