41–50 of 63 posts
In an age when your company mandates you to raise your productivity right now with hundreds of percentage points using LLMs, how do you find an excuse to sit down and read a book?
Earlier quoted context omitted.
It’s not about whether you work at Nvidia. Avoid writing CUDA kernels if there are higher level libraries that do what you need. Do write CUDA kernels if you want to learn how, or if you need the low level control, or to micro-optimize. Being able to fuse kernels to avoid memory traffic or get better specialization is also a reason to reach for raw CUDA. Just consider what’s the right tool for the job…
I don't think writing CUDA is a good way to do this tbh
Earlier quoted context omitted.
That advice seems like nonsense. It's like saying avoid C because you can use Python, or avoid writing a graphics engine because you can license Unreal.
Not at all, the advice is like use SDL or Raylib instead of writing your framebuffer blitter in inline Assembly to call from C.
Earlier quoted context omitted.
Not at all, the advice is like use SDL or Raylib instead of writing your framebuffer blitter in inline Assembly to call from C.
I bet you will learn alot doing that though
Each one has their place.
Earlier quoted context omitted.
the first book was published in 2012,is it too outdated?
Not really, Hardware didn't really change that much, of course you'll not find Tensor or raytracing cores, but you will have a very solid grasp of gpu programming and the cuda language (that didn't change that much either), and then you can easily learn those more modern things with blog posts or even, at worst, chatgpt.
I would separate the knowledge into maybe 3 distinct buckets.
The baseline: device/host boundary, SIMT programming etc.
The intermediate: kernel architecture, CUDA graph vs persistent kernels, warp specialisation/divergence avoidance techniques etc.
The advanced: architecture specifics so tcgen05, TMA, SMEM/HBM, memory throughput vs compute biases in various arch impls., GEMM, FHMA, all the tricks that make modern fused kernels very fast. Also would bucket most GPU Direct RDMA/GPU NetIO/friends here too.
The baseline hasn't changed much and probably won't, the intermediate knowledge has also remained pretty reliably stable for ~10 years with only things like graphs changing stuff. Tile might become more relevant than it is today but for now CUDA, cuBLAS, friends are where it's worth investing knowledge.
I wish the README had a solid “what cool things you can do with this” right at the top. In this day and age when programming is so accessible, why not have a more tempting pitch than just book titles categorized by difficulty.
With CUDA, you can make Nvidia GPUs go brrrr.
Oh. And thereby, incidentally conquer the compute world.
Earlier quoted context omitted.
I don't think writing CUDA is a good way to do this tbh
To do what? If you need the highest performance GPU kernel performance on NVidia HW, using CUDA is the way to go.
Earlier quoted context omitted.
Not really, Hardware didn't really change that much, of course you'll not find Tensor or raytracing cores, but you will have a very solid grasp of gpu programming and the cuda language (that didn't change that much either), and then you can easily learn those more modern things with blog posts or even, at worst, chatgpt.
Yeah pretty much this. I would separate the knowledge into maybe 3 distinct buckets. The baseline: device/host boundary, SIMT programming etc. The intermediate: kernel architecture, CUDA graph vs persistent kernels, warp specialisation/divergence avoidance techniques etc. The advanced: architecture specifics so tcgen05, TMA, SMEM/HBM, memory throughput vs compute biases in various arch impls., GEMM, FHMA, all the tri…
This would require very different (re-written?) kernels than a few years back, wouldn't it?
Would you have any good resources on the topic?
Any good MOOCs on Parallel programming/NVIDIA?