Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llm.c – LLM training in simple, pure C/CUDA

github.com

Post reply on HN
Print view View original

111–120 of 189 posts

Re: Llm.c – LLM training in simple, pure C/CUDA

#111

Post by xjay »

Art9681 wrote: post #103
ZoomerCretin wrote: post #62

Earlier quoted context omitted.

He loudly gave up on AMD after they did not fix a blocker he had for 5+ months and gave him the runaround the entire time when he asked for the code to fix it himself. He is still shipping the AMD tinybox with huge warning labels.

Didn't they recently announce that everything was open sourced? Would be cool if he took another look at it once all of the souce code is available (if not already).

> They haven't open sourced anything. They posted a tweet. [1]

[1@2024-04-06] https://www.youtube.com/watch?v=j7MRj4N2Cyk&t=429s

[Twitch] https://twitch.tv/georgehotz

Re: Llm.c – LLM training in simple, pure C/CUDA

#113

Post by hahnchen »

qwertox wrote: post #48

> direct CUDA implementation, which will be significantly faster and probably come close to PyTorch. It almost hurts, to read that PyTorch is faster. But then again, with these GPU-RAM-prices, let's see how it speeds up the CPU. We really need SO-DIMM slots on the RTX series (or AMD/Intel equivalent) so that we can expand the RAM as we need it to. Is there a technical problem to it?

> It almost hurts, to read that PyTorch is faster.

Why?

Re: Llm.c – LLM training in simple, pure C/CUDA

#114

Post by magicalhippo »

ZoomerCretin wrote: post #62
BraverHeart wrote: post #26

Earlier quoted context omitted.

George Hotz is attempting to solve this: https://github.com/tinygrad/tinygrad

He loudly gave up on AMD after they did not fix a blocker he had for 5+ months and gave him the runaround the entire time when he asked for the code to fix it himself. He is still shipping the AMD tinybox with huge warning labels.

Randomly stumbled over this[1] post with another fed up open source contributor, due to several serious issues with AMDs GPU drivers and firmware that remain unresolved for years. It also references the geohot decision you mention.

Some quotes:

I find it incredible that these companies that have large support contracts with you and have invested hundreds of thousands of dollars into your products, have been forced to turn to me, a mostly unknown self-employed hacker with very limited resources to try to work around these bugs (design faults?) in your hardware.

In the VFIO space we no longer recommend AMD GPUs at all, in every instance where people ask for which GPU to use for their new build, the advise is to use NVidia.

[1]: https://www.reddit.com/r/Amd/comments/1bsjm5a/letter_to_amd_...

Re: Llm.c – LLM training in simple, pure C/CUDA

#115

Post by auraham »

yinser wrote: post #37

I've seen his nano GPT implemented using JAX, now we have C/CUDA. I'd love to see if nano GPT could be doable in Mojo. I took a stab at a Mojo conversion of his Wavenet project (Andrej's zero to hero course) and I gotta say... python has so many nice features lol. Stating the obvious I know but what you see done in 6 lines of python takes so much more work in other languages.

Where is the GPT implementation in JAX? I only found this [1] in PyTorch and NumPy.

[1] https://github.com/karpathy/nanoGPT

Re: Llm.c – LLM training in simple, pure C/CUDA

#118

Post by cwegener »

andrewstuart wrote: post #22

OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?

Yes. CUDA is still entrenched due to its ease of use when compared to specialised AI hardware SDKs.

OpenMP is still a thing in 2024, but I presume that is not the kind of scale you are asking about.

If you have MI accelerators, you'll be using ROCm anyway (although AMD contracted Andrzej Janik in 2022 to make ZLUDA run on AMD GPUs. I have no idea what the practical applications of it are at the moment.)

The only other serious challenger (apart from the existing GPU manufactures like AMD) who is trying to give Nvidia a run for its money is Tenstorrent and their TT-Buda software kit.

Re: Llm.c – LLM training in simple, pure C/CUDA

#120

Post by idkwhatimdoin »

vineyardmike wrote: post #88
idkwhatimdoin wrote: post #64

If I was starting from scratch, what resources should I start with to build up an understanding of what this code does and how to read it? It's quite dense and my knowledge of LLMs is quite minimal. Are these terse variable names standard in LLM-land?

Terse variables are a C thing. “What resources would I need” -> you’re literally commenting on a teachers content. Karpathy (the author) has a very informative YouTube channel where he goes step by step through everything. He has a ton of repos and tutorials. Dig a little. If all else fails… Google it.

> you’re literally commenting on a teachers content.

How am I supposed to know that?

> Karpathy (the author) has a very informative YouTube channel where he goes step by step through everything.

Or that, without knowing that he's a teacher?

> Terse variables are a C thing.

I didn't realize variables had to be so short in C. Glad I write C++ professionally where they've added support for longer variable names.

> If all else fails… Google it.

There's a lot of LLM garbage out there. I got an answer here in a few minutes pointing to Karpathy's course which seems very high quality.

Be kinder.

Post reply on HN