I have fond memories of cs224d [1] taught by richardsocher. It’s a bit dated at this point as it was created in the pre-transformer era, but it was a very cool introduction to applying deep learning to nlp at the time. [1] https://cs224d.stanford.edu
CS336: Language Modeling from Scratch
11–20 of 56 posts
Re: CS336: Language Modeling from Scratch
#12> GPU compute for self-study Those suggestions they make for a B200 start at $4.99 an hour. Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai
Re: CS336: Language Modeling from Scratch
#13Re: CS336: Language Modeling from Scratch
#14> GPU compute for self-study Those suggestions they make for a B200 start at $4.99 an hour. Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai
Re: CS336: Language Modeling from Scratch
#15A want like a casual lesswrong style from ground up explanation.
Re: CS336: Language Modeling from Scratch
#16I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in
Would be great to have a community to discuss the material - even if folks can't commit to the full course.
Re: CS336: Language Modeling from Scratch
#17> GPU compute for self-study Those suggestions they make for a B200 start at $4.99 an hour. Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai
Re: CS336: Language Modeling from Scratch
#18> GPU compute for self-study Those suggestions they make for a B200 start at $4.99 an hour. Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai
It seems strange that the required resources aren't provided by the educational institution?
Re: CS336: Language Modeling from Scratch
#19> GPU compute for self-study Those suggestions they make for a B200 start at $4.99 an hour. Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai
We were lucky enough to get Blackwell GPUs for Stanford students this year, which is why the writeups are written mostly around them.
Re: CS336: Language Modeling from Scratch
#20AI Agent Guidelines for CS336 at Stanford https://github.com/stanford-cs336/assignment1-basics/blob/ma... (https://news.ycombinator.com/item?id=48359232)