Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
1–10 of 14 posts
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#2I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#3Very cool!!
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#4I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
Is your work publicly available anywhere? I'd love to have a look if possible.
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#5I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
Is your work publicly available anywhere? I'd love to have a look if possible.
Hi :) Yes it's in the post url or here: https://github.com/mlecauchois/micrograd-cuda
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#6I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
Awesome, thank you for publishing it. I was planning the same, but didn't find the time yet.
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#7Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#8Cool
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#9I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
Lovely project, what did you find as the beat place to learn about cuda?
Re: Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
#10I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd ( https://github.com/karpathy/micrograd ), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM