The Smol Training Playbook: The Secrets to Building World-Class LLMs
1–10 of 20 posts
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#2Hugging Face is not just an AI information-sharing website; it’s also a great learning platform for all AI learners. This documentation is one of the most impressive hands-on resources I’ve ever read.
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#3Hugging Face is not just an AI information-sharing website; it’s also a great learning platform for all AI learners. This documentation is one of the most impressive hands-on resources I’ve ever read.
What others would you recommend that are comparable in quality?
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#4Hugging Face is not just an AI information-sharing website; it’s also a great learning platform for all AI learners. This documentation is one of the most impressive hands-on resources I’ve ever read.
What others would you recommend that are comparable in quality?
The documentation for common ai packages is pretty good too. For example, pytorch docs, peft docs, timm docs.
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#5Hi, Lewis here (one of the co-authors). Happy to answer any questions people have about the book :)
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#6Hi, Lewis here (one of the co-authors). Happy to answer any questions people have about the book :)
I'm a little ways through this and it's great so far, nice job.
One of the reasons people build one though is to learn. Most smart folks are quite aware that the reality of pre-training a real LLM is going to involve some head banging against the wall (ie, things don't go smoothly like "building an llm from scratch" book), and they want to go through the process.
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#7Hugging Face is not just an AI information-sharing website; it’s also a great learning platform for all AI learners. This documentation is one of the most impressive hands-on resources I’ve ever read.
What others would you recommend that are comparable in quality?
Been reading a book by u/fpham "The Cranky mans guide to lora and qlora" and it's pretty great, writing quality isnt all there but the content is valuable for learning to make good finetunes
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#8Where does "Smol" come from? It's supposed to mean "Small" right? If yes then what's the etymology and reason for popular usage?
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#9I really like the Hugging Face guys, but...
> Modify one thing at a time
> Change only one variable per ablation while keeping everything else constant. If you change multiple things and performance improves, you won’t know what caused it. Test modifications individually, then combine successful ones and reassess.
This is an unintentional microcosm of what is flawed with the document.
Re: The Smol Training Playbook: The Secrets to Building World-Class LLMs
#10I really like the Hugging Face guys, but... > Modify one thing at a time > Change only one variable per ablation while keeping everything else constant. If you change multiple things and performance improves, you won’t know what caused it. Test modifications individually, then combine successful ones and reassess. This is an unintentional microcosm of what is flawed with the document.
What's wrong with it? That's good advice in almost any optimization or troubleshooting context where variables may interact.