Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration

Post by cataPhil » Thu, Mar 21, 2024, 8:20 AM UTC

lairv wrote: post #10

cataPhil wrote: post #2

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM

Post by st380752143 » Thu, Mar 21, 2024, 11:14 PM UTC

cataPhil wrote: post #11

coolThingsFirst wrote: post #9

Earlier quoted context omitted.

Lovely project, what did you find as the beat place to learn about cuda?

Thanks! I highly recommend the book Programming Massively Parallel Processors