Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
magazine.sebastianraschka.com
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
1–10 of 29 posts
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#2Ever since the author paywalled some of his useful posts, I stopped following him. I have read his ML book and I know he used to be a professor and is now working in the industry, and he’s quite famous in the field. That’s why I don’t understand why such a figure would even need the extra income generated by Substack’s paywall.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#3LoRA blew me away the first time I looked into it. Especially since you can host many LoRA adapters at once for a fraction of the cost of hosting an entire model by sharing the base between the adapters. I built a little tool to make LoRA fine-tuning easier. The adapters export to Huggingface. You can check it out here: https://app.haven.run
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#4Ever since the author paywalled some of his useful posts, I stopped following him. I have read his ML book and I know he used to be a professor and is now working in the industry, and he’s quite famous in the field. That’s why I don’t understand why such a figure would even need the extra income generated by Substack’s paywall.
That’s a strange question. He wants to make money doing what he loves - teaching about AI. What’s wrong with that?
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#5Ever since the author paywalled some of his useful posts, I stopped following him. I have read his ML book and I know he used to be a professor and is now working in the industry, and he’s quite famous in the field. That’s why I don’t understand why such a figure would even need the extra income generated by Substack’s paywall.
looks like this one isn't paywalled (for me at least)
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#6Ever since the author paywalled some of his useful posts, I stopped following him. I have read his ML book and I know he used to be a professor and is now working in the industry, and he’s quite famous in the field. That’s why I don’t understand why such a figure would even need the extra income generated by Substack’s paywall.
looks like this one isn't paywalled (for me at least)
This isn't, but some other articles are.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#7Ever since the author paywalled some of his useful posts, I stopped following him. I have read his ML book and I know he used to be a professor and is now working in the industry, and he’s quite famous in the field. That’s why I don’t understand why such a figure would even need the extra income generated by Substack’s paywall.
That’s a strange question. He wants to make money doing what he loves - teaching about AI. What’s wrong with that?
Nothing wrong with that. But it's strange that such a person (who undoubtedly makes $$$) wants to make some more $×10^-n (n ≥1) by paywalling his articles.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#8Earlier quoted context omitted.
That’s a strange question. He wants to make money doing what he loves - teaching about AI. What’s wrong with that?
Nothing wrong with that. But it's strange that such a person (who undoubtedly makes $$$) wants to make some more $×10^-n (n ≥1) by paywalling his articles.
> (who undoubtedly makes $$$)
He worked at a public university until 2021, you can look up his salary as it's public information: $118,472.99 [0], not as much as your average mid-level software engineer.
Now he works at a startup [1], but not as a c-level, so he's likely making average startup software engineer salary (certainly more than a public university professor, but not exactly FIRE money).
It's amazing how much people begrudge anyone wanting to make some money for the tremendous effort the put in to helping people better understand an important subject. Anyone working in software can easily afford to support his work if they find it valuable.
Generally, anyone "famous" for teaching a technical topic is typically doing it as a charity even if people are paying for their content.
0. https://madison.com/uw-salary-database/html_e37ff025-9a87-5a... 1. https://sebastian.com
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#9I've been increasingly wondering if the field considering LLMs as a continuum as opposed to a set of distinct thresholds is leading to erroneous "rules of thumb" as most research on methodology is concentrated in smaller and more accessible model experimentation right now.
We generally recognize (nearly ad nauseum) that mouse models of medical research don't necessarily translate to humans.
Similarly, I'd imagine most would laugh at the idea that a neurology researcher who found the best way to get a fruit fly's brain to navigate a maze should extrapolate that methodology to a dolphin or a chimp's brain.
Maybe we should be defining "weight classes" for LLMs and grouping research based on those classes. Like "these are the techniques that work best for lightweight models" but not necessarily assuming those as a general rule of thumb for "heavyweight models."
Even something like the discussion of synthetic data on model collapse is a good example of where there might be a very significant difference in the effect on model quality for a cheaper and less sophisticated model generating synthetic data to feed back into itself and a much more complex and sophisticated model. Maybe the lesson is actually "recursive training on synthetic data leads to model collapse in lightweight and medium weight models."
So while the writeup is a great one on fine tuning 7B models with LoRA, I would be curious just what % of the recommendations hold true in replication for even just a 65B model.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#10LoRA blew me away the first time I looked into it. Especially since you can host many LoRA adapters at once for a fraction of the cost of hosting an entire model by sharing the base between the adapters. I built a little tool to make LoRA fine-tuning easier. The adapters export to Huggingface. You can check it out here: https://app.haven.run
What software do you use for the lora sharing?