I'm currently going over the lectures and assignments of this course on the side of work. And I can only recommend it, it is great quality and the 2026 version is really up to date with architecture decisions. I'm really enjoying it.
And I also appreciate the low compute tips to run things on mac
CS336: Language Modeling from Scratch
51–56 of 56 posts
Re: CS336: Language Modeling from Scratch
#52i recently started reading "build reasoning model from scratch" then i realized that i am not really interested in building part and just want to understand theory and practice behind it. A want like a casual lesswrong style from ground up explanation.
In that case I humbly suggest my talk from AI Engineer World's Fair https://www.youtube.com/watch?v=ZuiJjkbX0Og Gives you the basics on LLM internals in about 90 minutes and includes an already built model in JavaScript that you can step through in browser devtools to get as detailed as you want.
Thanks for linking to it! Just in time for my own learner's journey.
Re: CS336: Language Modeling from Scratch
#53[dead]
Re: CS336: Language Modeling from Scratch
#54Is it possible to complete assignments with AMD GPUs? Preferably on Windows.
Re: CS336: Language Modeling from Scratch
#55I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in
I would be interested in this. I opened this thread to see if anyone posted about doing the course as a group!
Re: CS336: Language Modeling from Scratch
#56I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in
I'd be interested in joining a discord server. Would be great to have a community to discuss the material - even if folks can't commit to the full course.
+1, did you end up creating a discord server?