Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyLoRA – Learning to Reason in 13 Parameters

arxiv.org

Post reply on HN
Print view View original

21–30 of 54 posts

Re: TinyLoRA – Learning to Reason in 13 Parameters

#21

Post by robrenaud »

dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

Yeah, my big problem with the paper is it just might be an artifact of qwen's training process.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#22

Post by sachaa »

dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

Fair points, especially on GSM8K saturation and Qwen possibly already sitting close to the solution. That said, even if this is mostly "last-mile alignment", the fact that it can be done with such a tiny signal is still interesting, it suggests the gap between capability and behavior might be much smaller (and cheaper to bridge) than we assume.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#24

Post by Xx_crazy420_xX »

If i understand it correctly, the analogy could be:

Let's say we have a low level programmer expert and we try to teach him algebra either we:

  - (SFT): give him algebra book with new nomenclature, definitions, syntax
  - (RL): let him learn algebra using C syntax

Re: TinyLoRA – Learning to Reason in 13 Parameters

#26

Post by taneq »

robrenaud wrote: post #21
dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

Yeah, my big problem with the paper is it just might be an artifact of qwen's training process.

In all fairness most of the unique stuff I can do is probably an artifact of my training process, so it seems unfair to deny an LLM the same accomodation.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#29

Post by nathan_compton »

Xx_crazy420_xX wrote: post #24

If i understand it correctly, the analogy could be: Let's say we have a low level programmer expert and we try to teach him algebra either we: - (SFT): give him algebra book with new nomenclature, definitions, syntax - (RL): let him learn algebra using C syntax

I don't think so.

Fine tuning works on an input/output basis. You are rewarded for producing a plausible output _now_.

RL rewards you later for producing the right output now. So you have to learn to generate a lot of activity but you are only rewarded if you end up at the right place.

In SFT you are rewarded for generating tokens plausible to the proof, one token at a time. In RL you are expected to generate an entire proof and then you are rewarded or punished only when the proof is done.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#30

Post by sorenjan »

dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

They're using the truncated SVD, not the full variant, that's computationally cheaper.
Post reply on HN