What every developer should know about GPU computing • Hacker News Board

Re: What every developer should know about GPU computing

#181

Post by [deleted] » Tue, Oct 24, 2023, 6:33 AM UTC

winwang wrote: post #177

subharmonicon wrote: post #156

Earlier quoted context omitted.

Can you cite a reference explaining this ability to re-form new warps from existing threads? I ask because I’ve seen posts from NVIDIA support saying that divergence is still very expensive and I’ve also seen benchmarks that force divergence in each warp by evenly splitting the warp, and the benchmarks result in 2x runtime when that happens vs. when the control-flow is dynamically uniform. One thing to keep in mind i…

"Divergence is still very expensive" is quite compatible with "Divergence is less expensive than before". Here's evidence (not proof) that Nvidia would remove the hard limit of warp divergence (or perhaps more "precisely", *a warp is always synchronous across its 32 threads with divergence"): https://developer.nvidia.com/blog/cooperative-groups/ I don't think it's misleading to talk about a "CUDA core" as a warp-wide…

[deleted]

Re: What every developer should know about GPU computing

#182

Post by freeone3000 » Tue, Oct 24, 2023, 6:39 AM UTC

winwang wrote: post #177

subharmonicon wrote: post #156

Earlier quoted context omitted.

Can you cite a reference explaining this ability to re-form new warps from existing threads? I ask because I’ve seen posts from NVIDIA support saying that divergence is still very expensive and I’ve also seen benchmarks that force divergence in each warp by evenly splitting the warp, and the benchmarks result in 2x runtime when that happens vs. when the control-flow is dynamically uniform. One thing to keep in mind i…

"Divergence is still very expensive" is quite compatible with "Divergence is less expensive than before". Here's evidence (not proof) that Nvidia would remove the hard limit of warp divergence (or perhaps more "precisely", *a warp is always synchronous across its 32 threads with divergence"): https://developer.nvidia.com/blog/cooperative-groups/ I don't think it's misleading to talk about a "CUDA core" as a warp-wide…

the SIMT section there is pretty telling — you can do it, if you explicitly account for it, and are willing to leave other threads in the dust potentially forever. It’s not quite the same thing as JMP and only seems to account for the data dependency case and not the if/else non-stream case.

But I stand corrected — Volta and up have multiple PCs per warp.

Re: What every developer should know about GPU computing

#183

Post by protomolecule » Tue, Oct 24, 2023, 2:19 PM UTC

jandrewrogers wrote: post #173

pca006132 wrote: post #159

Earlier quoted context omitted.

I am really curious about what kind of optimizations are enabled when you know the microarchitectural design of various CPUs, but without writing assembly by hand. I only know the basics such as optimizing data structure for better cache locality, add some fast paths, manual unrolling, but have no idea about how to work around things like pipeline stalls. It would be really helpful if you can point to some materials…

The basic principle is that a CPU core is a complex distributed system with varying degrees of parallelism, concurrency, and latency as all of these components communicate with each other. The design of this distributed system varies across microarchitectures as do the available features. Analyzing the optimal codegen is similar to analyzing optimal algorithm design in higher level distributed systems with a fixed ha…

"There is an active niche community on the Internet that studies how various instruction sequences interact with various microarchitectures. This is probably the best resource"

Could elaborate on that? I mean where do I find this community?

Re: What every developer should know about GPU computing

#184

Post by [deleted] » Tue, Oct 24, 2023, 9:27 PM UTC

jacquesm wrote: post #107

fatih-erikli wrote: post #92

I think GPU computing should not be done in application layer. It's way too low-level.

That's the only layer where it makes sense because that's where you know what it is that you are trying to achieve. The overhead in GPU programming is such that if you make one small assumption that doesn't hold true in practice you may end up sinking your performance in a terrible way. So you need a lot of control over where and how things are laid out. For more generic stuff there are libraries, but those too run a…

[deleted]

Re: What every developer should know about GPU computing

#185

Post by fatih-erikli » Tue, Oct 24, 2023, 9:32 PM UTC

jacquesm wrote: post #107

fatih-erikli wrote: post #92

I think GPU computing should not be done in application layer. It's way too low-level.

That's the only layer where it makes sense because that's where you know what it is that you are trying to achieve. The overhead in GPU programming is such that if you make one small assumption that doesn't hold true in practice you may end up sinking your performance in a terrible way. So you need a lot of control over where and how things are laid out. For more generic stuff there are libraries, but those too run a…

Makes sense.

Re: What every developer should know about GPU computing

#186

Post by nn_m » Thu, Oct 26, 2023, 3:09 PM UTC

PTOB wrote: post #53

permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

Try this on: "A non-trivial number of Computer Scientists, Computer Engineers, Electrical Engineers, and hobbyists have ..." Took some philosophy courses for fun in college. I developed a reading skill there that lets me forgive certain statements by improving them instead of dismissing them. My brain now automatically translates over-generalizations and even outright falsehoods into rationally-nearby true statements…

Thanks so much for sharing this. I'm watching a self described "politicsl philosopher" on yTube. (The talks are labeled as chats, are not scripted, although he has separate channel that offers scripted material)

The "chats" are usually edifying, but I am often frustrated by the rambling and repetition (He is always trying to bring in new listeners, so that partly explains the repetition)

But from my own (arm chair politcal organizer) point of view, i'm thinking "you have ~10 very good key points you touch on/repeat over time, why not be a more effective communicator, make this visual as well as verbal and turn this into a message. I.e. bullet points that are always close at hand and that you're l using to tie the talk together."

But that's basically propaganda main tool.

So thanks for sharing your effort and discipline in going to the philosophy classes. Your more expansive approach to what people aay chimes very much with my yT mentor's reminder to listen 99% and judge 1%.

Dont try to win every argument or prove your point of view is right. Others are entitled to their (wrong (-; ) opinions (just as if we think back to our changing points of view, we have been "wrong" in the past too. Judge yee not ....)

By continuing to listen to those we don't agree with we are putting a face to ideas they dont accept or understand (it is easier to stay in your own bubble when you are only looking/hearing people that you agree with) AND by listening respectfully we are keeping the lines of communication open.