1–10 of 52 posts
Seems like ML is taking off recently and I want to get back into it! So far on my list I have attention is all you need, qlora, llama’s and q learning. Suggestions?
This is a great little book to take you from “vaguely understand neural networks” to the modern broad state of practice. I saw very little to quibble with. https://fleuret.org/francois/lbdl.html
Maybe this tweet by John Carmack can help you: This is a great little book to take you from “vaguely understand neural networks” to the modern broad state of practice. I saw very little to quibble with. https://fleuret.org/francois/lbdl.html
And, Francois could easily report the unauthorized seller to Amazon, or send S&D letter, suing not required.
[1] https://simonwillison.net/2023/Aug/3/weird-world-of-llms/ [2] https://youtu.be/zjkBMFhNj_g?si=M6pRX66NrRyPM8x-
EDIT: Maybe I misunderstood as you asked about papers, not general intros. I don´t think that reading papers is the best way to "catch up" as the pace is rapid and knowledge very decentralized. I can confirm what Andrej recently wrote on X [3]:
"Unknown to many people, a growing amount of alpha is now outside of Arxiv, sources include but are not limited to:
- HN
- that niche Discord server
- anime profile picture anons on X
- reddit"
This is a good explanation of the Transformer details -> https://www.youtube.com/watch?v=bCz4OMemCcA&ab_channel=UmarJ...
This is old but covers a lot of background that you needs to know to understand very well the rest. What I like of this book is that it often explains in a very intuitive way the motivations behind certain choices. -> https://www.amazon.it/Natural-Language-Processing-Pytorch-Ap...
Here's also nice tour de building blocks, which could also double as transformers/tensorflow API reference documentation: https://www.youtube.com/watch?v=eMXuk97NeSI&t=207s
The #1 visualization of architecture and size progression: https://bbycroft.net/llm
Foundational model training got so expensive that unless you can get hired by "owns nuclear power plant of GPUs" you are not going to get any "research" done. And as the area got white-hot those companies have more available talent than hardware nowadays. So just getting into the practitioner area is the best way to get productive with those models. And you improve as a practitioner by practicing, not by reading papers.
If you're at the computer, your time is best spent writing code and interacting with those models in my opinion. If you cannot (e.g. commute) I listen to some stuff (e.g. https://www.youtube.com/watch?v=zjkBMFhNj_g - Anything from Karpathy on youtube, or https://www.youtube.com/@YannicKilcher channel).
http://bactra.org/notebooks/nn-attention-and-transformers.ht...
Definitely read through to the last section.