CS336: Language Modeling from Scratch

#1

Post by kristianpaul » Mon, Jun 01, 2026, 2:10 PM UTC

CS336: Language Modeling from Scratch
cs336.stanford.edu

Re: CS336: Language Modeling from Scratch

#2

Post by storus » Mon, Jun 01, 2026, 3:24 PM UTC

Thanks for releasing this again! What are this year's changes to prior offerings?

Re: CS336: Language Modeling from Scratch

#3

Post by meken » Mon, Jun 01, 2026, 3:38 PM UTC

I have fond memories of cs224d [1] taught by richardsocher. It’s a bit dated at this point as it was created in the pre-transformer era, but it was a very cool introduction to applying deep learning to nlp at the time.

[1] https://cs224d.stanford.edu

Re: CS336: Language Modeling from Scratch

#4

Post by tmule » Mon, Jun 01, 2026, 3:39 PM UTC

Are video lectures available online?

Re: CS336: Language Modeling from Scratch

#5

Post by Bilal_io » Mon, Jun 01, 2026, 3:44 PM UTC

tmule wrote: post #4

Are video lectures available online?

Youtube playlist link from the page https://www.youtube.com/watch?v=JuoVZkPBiKk&list=PLoROMvodv4...

Re: CS336: Language Modeling from Scratch

#6

Post by [deleted] » Mon, Jun 01, 2026, 3:44 PM UTC

tmule wrote: post #4

Are video lectures available online?

[deleted]

Re: CS336: Language Modeling from Scratch

#7

Post by mindcrime » Mon, Jun 01, 2026, 3:45 PM UTC

tmule wrote: post #4

Are video lectures available online?

https://www.youtube.com/playlist?list=PLoROMvodv4rMqXOcazWaT...

Re: CS336: Language Modeling from Scratch

#8

Post by aerohit » Mon, Jun 01, 2026, 3:46 PM UTC

tmule wrote: post #4

Are video lectures available online?

https://www.youtube.com/watch?v=JuoVZkPBiKk&list=PLoROMvodv4...

Re: CS336: Language Modeling from Scratch

#9

Post by skerit » Mon, Jun 01, 2026, 4:07 PM UTC

> GPU compute for self-study

Those suggestions they make for a B200 start at $4.99 an hour.

Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai

Re: CS336: Language Modeling from Scratch

#10

Post by airstrike » Mon, Jun 01, 2026, 4:25 PM UTC

I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in