It would be great if someone created a tutorial around this explaining exactly how it works and how to do a test training run. I’m aware it’s not feasible to train a “real” model on personal hardware but it would be nice to have a practical learning experience. I’m not sure if there are good alternatives for that.
Llm.c – LLM training in simple, pure C/CUDA
41–50 of 189 posts
Re: Llm.c – LLM training in simple, pure C/CUDA
#42> LLM training in simple, pure C/CUDA. There is no need for 245MB of PyTorch or 107MB of cPython
[deleted]
Re: Llm.c – LLM training in simple, pure C/CUDA
#43[flagged]
[deleted]
Re: Llm.c – LLM training in simple, pure C/CUDA
#44Earlier quoted context omitted.
107MB of cPython defeated Go to try for self Step 1 download 2.4GB of CUDA
The size of CUDA really is astonishing. Any chance someone might figure out how to slim that down?
Raise your voice on their forum: https://forums.developer.nvidia.com/t/how-to-overcome-the-hu...
Tried my luck 2 years ago but it keeps increasing.
Re: Llm.c – LLM training in simple, pure C/CUDA
#45[flagged]
[deleted]
Re: Llm.c – LLM training in simple, pure C/CUDA
#46OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?
There are a few attempts here and there in various stages of progression. But right now, nothing matches Nvidia+CUDA in speed and usability.
Re: Llm.c – LLM training in simple, pure C/CUDA
#47Earlier quoted context omitted.
107MB of cPython defeated Go to try for self Step 1 download 2.4GB of CUDA
The size of CUDA really is astonishing. Any chance someone might figure out how to slim that down?
Here’s a blog that breaks down how large different pieces of CUDA are:
Re: Llm.c – LLM training in simple, pure C/CUDA
#48> direct CUDA implementation, which will be significantly faster and probably come close to PyTorch.
It almost hurts, to read that PyTorch is faster.
But then again, with these GPU-RAM-prices, let's see how it speeds up the CPU.
We really need SO-DIMM slots on the RTX series (or AMD/Intel equivalent) so that we can expand the RAM as we need it to. Is there a technical problem to it?
Re: Llm.c – LLM training in simple, pure C/CUDA
#49Re: Llm.c – LLM training in simple, pure C/CUDA
#50https://twitter.com/karpathy/status/1777427944971083809
> And once this is a in a bit more stable state: videos on building this in more detail and from scratch.
Looking forward to watching the videos.