Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyLoRA – Learning to Reason in 13 Parameters

arxiv.org

Post reply on HN
Print view View original

41–50 of 54 posts

Re: TinyLoRA – Learning to Reason in 13 Parameters

#41

Post by endofreach »

sachaa wrote: post #22
dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

Fair points, especially on GSM8K saturation and Qwen possibly already sitting close to the solution. That said, even if this is mostly "last-mile alignment", the fact that it can be done with such a tiny signal is still interesting, it suggests the gap between capability and behavior might be much smaller (and cheaper to bridge) than we assume.

> the gap between capability and behavior might be much smaller

Can you elaborate a bit on what you mean with the gap?

Re: TinyLoRA – Learning to Reason in 13 Parameters

#42

Post by crawfordcomeaux »

a-t-c-g wrote: post #15
matt123456789 wrote: post #13

Such low dimensionality of the LoRA vector must surely result in a close-to-linear modification to the KV calculation. This seems to me to imply that what we call "reasoning" is latent within the model. Pretty clear I didn't read the paper, I'm sure the authors address this.

Yes - some degree of reasoning appears to be latent in the structure of language itself. But models trained explicitly on reasoning-focused data still perform better than models trained only on general corpora.* *At least up to 300B parameters, based on the models we’ve tested.

I wonder what the relationships between the grammar of a language, what it can compute, how it encodes, and what the minimal parameters/structure for reasoning looks like...

Re: TinyLoRA – Learning to Reason in 13 Parameters

#43

Post by cheald »

dollo_7 wrote: post #19

Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models. But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to ha…

I've done a lot of exploratory work with Stable Diffusion LoRAs, and I actually do buy that there's some juice here, though it's almost certainly not nearly as good as other techniques can be. In particular, this technique will likely avoid the intruder dimension problem which plagues naive LoRA. SVD is expensive, but you only have to do it once at the beginning of training.

I haven't done much research lately, but when I was working on it, I was having substantial success training an adapter of the form U_k @ P @ A, where U_k was the top k left singular vectors of the underlying weight, and then P and A were your typical LoRA projection matrices.

The 13 parameters are kind of misleading here; the real juice is going to be in the P_i fixed random matrices. My suspicion is that they are overfitting to the benchmark, but they almost certainly are observing a real gain in model capacity that is largely due to avoiding the intruder dimension problem.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#44

Post by HeyLaughingBoy »

ticulatedspline wrote: post #39
jorlow wrote: post #37

Earlier quoted context omitted.

I see this comment on every single LoRA post despite the vast majority of posts being about LoRA not LoRa. Can we please stop beating this dead horse?

Never heard of the radio thing. I suspect LoRA has already eclipsed LoRa in general usage. It's probably more appropriate to complain on a LoRa post that it's too close to LoRA.

And I'm the exact opposite. I never heard about LoRA, but I have used LoRa and was curious to see what it had to do with reasoning.

It's just an unfortunate name collision: disambiguating by use of capitals only works with computers.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#45

Post by a-t-c-g »

crawfordcomeaux wrote: post #42
a-t-c-g wrote: post #15

Earlier quoted context omitted.

Yes - some degree of reasoning appears to be latent in the structure of language itself. But models trained explicitly on reasoning-focused data still perform better than models trained only on general corpora.* *At least up to 300B parameters, based on the models we’ve tested.

I wonder what the relationships between the grammar of a language, what it can compute, how it encodes, and what the minimal parameters/structure for reasoning looks like...

natural language may provide part of the scaffolding for reasoning, but the capability itself seems to depend more on learned transformations over internal representations than on language alone

refs: https://arxiv.org/abs/2412.17819 https://arxiv.org/abs/2412.06769

Re: TinyLoRA – Learning to Reason in 13 Parameters

#46

Post by msdz »

nativeit wrote: post #33
taneq wrote: post #26

Earlier quoted context omitted.

In all fairness most of the unique stuff I can do is probably an artifact of my training process, so it seems unfair to deny an LLM the same accomodation.

How much did your training cost society?

This got me thinking, and it might actually even be a comparable amount. Let's estimate 12 years of schooling run at minimum $100,000 per student, at least in the US [1], and then add onto that number whatever else you may do after that, i.e. a bunch more money if paid (college) or "unpaid" (self-taught skills and improvements) education, and then the likely biggest portion for white-collar workers, yet hard-to-quantify, in experience and "value" professional work will equip one with.

Now divide the average SOTA LLM's training cost (or a guess, since these numbers aren't always published as far as I'm aware) by the number of users, or if you wanted to be more strict, the number of people it's proven to be useful for (what else would training be for), and it might not be so far off anymore?

Of course, whether it makes sense to divide and spread out the LLMs' costs across users in order to calculate an "average utility" is debatable.

[1] https://www.publicschoolreview.com/average-spending-student-...

Re: TinyLoRA – Learning to Reason in 13 Parameters

#48

Post by a-t-c-g »

objektif wrote: post #31
a-t-c-g wrote: post #14

Earlier quoted context omitted.

Benchmarks, we have internal ones testing reasoning fine-tuned v/s frontier + prompts For some use cases it can be parity performance at 1/20th the cost up to exceeds at 1/10th the cost. Trade-off is ofc narrow applicability

How can I learn more about these models? Are they open source?

there are plenty of OSS finetuned models + base models around. If you're looking for doing these on your own dataset, worth getting in touch with cartesien.io or wire up https://github.com/SalesforceAIResearch/PretrainRL-pipeline

Re: TinyLoRA – Learning to Reason in 13 Parameters

#50

Post by objektif »

a-t-c-g wrote: post #48
objektif wrote: post #31

Earlier quoted context omitted.

How can I learn more about these models? Are they open source?

there are plenty of OSS finetuned models + base models around. If you're looking for doing these on your own dataset, worth getting in touch with cartesien.io or wire up https://github.com/SalesforceAIResearch/PretrainRL-pipeline

Thank you.
Post reply on HN