The name gets me every single time. Always think it’s going to be about radio LoRa
Not just me then. It's always the first thing that springs to mind.
LoRA Without Regret
21–30 of 62 posts
Re: LoRA Without Regret
#22Can someone explain the bit counting argument in the reinforcement learning part? I don’t get why a trajectory would provide only one bit of information. Each step of the trajectory is at least giving information about what state transitions are possible. An infinitely long trajectory can explore the whole state space if there are no absorbing states. Such a trajectory would provide a massive amount of information ab…
A fair amount of research has shown that RL doesn’t add knowledge to the base model it just optimizes paths that already exist.
Now ProRL from Nvidia showed there are ways of adding knowledge, mostly through progressive merging.
I’m still not fully convinced of the 1bit claim, they made other mistakes in the blog post
Re: LoRA Without Regret
#23> LoRA works well when not capacity constrained, i.e., the number of trainable parameters exceeds the amount of information to be learned, which can be estimated in terms of dataset size
I’m shocked they didn’t look at progressive merging of LoRAs. Research shows that’s the best way of improving its ability to model higher level features.
Seems like a massive miss, not to mention there is other research that contradicts a lot of their findings. This feels a bit like a researchers first pass at learning LoRA
Re: LoRA Without Regret
#24Thinking Machines have put out a string of incredibly high-quality posts lately. Hard to oversell how much cred it's buying them with the AI research community! Keep up the great work folks
What else has there been. I've only seen this one (which is great!)
Their Defeating Nondeterminism in LLM Inference was interesting for me. Worth reading their others!
Re: LoRA Without Regret
#25The amount of people who mistook this for long range radio and were disappointed when it isnt about it is way too damn high. (This is including me)
Re: LoRA Without Regret
#26> However, the literature is unclear on how well LoRA performs relative to FullFT.
I think the literature is clear on that?
"LoRA vs Full Fine-tuning: An Illusion of Equivalence" -- https://arxiv.org/abs/2410.21228v1
Quoting from the conclusions:
> The paper describes the finding that LoRA and full fine-tuning, with equal performance on the fine-tuning task, can have solutions with very different generalization behaviors outside the fine-tuning task distribution. We found that LoRA and full fine-tuning yield models with significant differences spectral properties of their weight matrices: LoRA models often containing “intruder dimensions”, high-ranking singular vectors approximately orthogonal to the singular vectors of pre-trained weight matrices. The existence of intruder dimensions correlates with the fine-tuned model forgetting more of the pre-training distribution as well as forgetting more when trained on tasks sequentially in a continual learning setup.
I'm surprised they didn't cite this; it's a well known paper.
Re: LoRA Without Regret
#27The amount of people who mistook this for long range radio and were disappointed when it isnt about it is way too damn high. (This is including me)
It might be useful to use this thread in a dataset to train a LoRa so that LLM agents can more easily disambiguate the great LoRa acronym collision of ‘25. No longer will future generations suffer the indignity of either/or/both confusions.
Re: LoRA Without Regret
#28> However, the literature is unclear on how well LoRA performs relative to FullFT. I think the literature is clear on that? "LoRA vs Full Fine-tuning: An Illusion of Equivalence" -- https://arxiv.org/abs/2410.21228v1 Quoting from the conclusions: > The paper describes the finding that LoRA and full fine-tuning, with equal performance on the fine-tuning task, can have solutions with very different generalization behav…
To say that the 'literature is clear on that' while citing a single paper, which has been rejected from ICLR, is a bit of an overstatement.
Re: LoRA Without Regret
#29Re: LoRA Without Regret
#30> However, the literature is unclear on how well LoRA performs relative to FullFT. I think the literature is clear on that? "LoRA vs Full Fine-tuning: An Illusion of Equivalence" -- https://arxiv.org/abs/2410.21228v1 Quoting from the conclusions: > The paper describes the finding that LoRA and full fine-tuning, with equal performance on the fine-tuning task, can have solutions with very different generalization behav…
To say that the 'literature is clear on that' while citing a single paper, which has been rejected from ICLR, is a bit of an overstatement.
Thanks for this comment.