शगहस
ह्षब
डीबीएनएस
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
21–29 of 29 posts
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#22I'm still waiting for someone to publish a "LoRA in ten steps" document, with working code, aimed at impatient people like me.
Then put that in a docker container for people like me who are too impatient to read 10 step documents.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#23Earlier quoted context omitted.
Nothing wrong with that. But it's strange that such a person (who undoubtedly makes $$$) wants to make some more $×10^-n (n ≥1) by paywalling his articles.
> (who undoubtedly makes $$$) He worked at a public university until 2021, you can look up his salary as it's public information: $118,472.99 [0], not as much as your average mid-level software engineer. Now he works at a startup [1], but not as a c-level, so he's likely making average startup software engineer salary (certainly more than a public university professor, but not exactly FIRE money). It's amazing how mu…
When he was professor at a public university he only had twice the median income of people in his State.
That greediness is begrudged is ok imho, but we should start with bigger fishes
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#24Earlier quoted context omitted.
That’s a strange question. He wants to make money doing what he loves - teaching about AI. What’s wrong with that?
Nothing wrong with that. But it's strange that such a person (who undoubtedly makes $$$) wants to make some more $×10^-n (n ≥1) by paywalling his articles.
More $$$ -> more peace of mind for open-ended content and research.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#25शगहस ह्षब डीबीएनएस
@dang seems like bot/spam account. Can you ban it ?
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#26what is the toolset that works best ?
axolotl is generally recommended...but unsure if that is what is genuinely the best for production scale finetuning.
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#27I fine tuned LLama-2 on code/comment generation (in python) for around $2 and was able to run it natively on an m1 macbook air. I can totally see smaller fine tuned LLM's being used locally on consumer devices in the future. I think people underestimate how cheap and efficient this stuff is. I've actually built a service which lets you fine tune LLama-2/other llms by uploading a JSON dataset. I'm looking for feedback…
I've been playing with llama-2, and I've been pleasantly surprised with its ability to process images. So you plan to offer image inputs on your fine tuning service?
That sounds interesting. Do you have more info on how llama2 can process images?
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#28Earlier quoted context omitted.
I've been playing with llama-2, and I've been pleasantly surprised with its ability to process images. So you plan to offer image inputs on your fine tuning service?
That sounds interesting. Do you have more info on how llama2 can process images?
I've been using this [0] locally, and this model [1] on replicate.
[0] https://github.com/ggerganov/llama.cpp
[1] https://replicate.com/yorickvp/llava-13b/versions/6bc1c7bb0d...
Re: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
#29I’ve been thinking about ways to compress information with ai for long distance transmission with LoRA radio for a while and now this LoRA in the news gets me all confused.