TinyLoRA – Learning to Reason in 13 Parameters • Hacker News Board

Re: TinyLoRA – Learning to Reason in 13 Parameters

#51

Post by 5555watch » Thu, Apr 02, 2026, 1:59 AM UTC

It's not "13 parameters to reason", they just rotated the full 8B parameter space in 13 dimensions and found a rotation that was still able to reason.

Depending on the latent structure, it's possible a nice rotation that would be perfect for some one specific problem, but you still got to search for it, and it's not a guarantee to exist.

But it's a nice step towards LLM parameter-space interpretability.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#52

Post by voxelghost » Thu, Apr 02, 2026, 2:56 AM UTC

volume_tech wrote: post #32

[flagged]

Thats a wonderful explanation (and roughly the conclussion I arrived at after browsing the paper), I just wish it would have been in the original post.

Re: TinyLoRA – Learning to Reason in 13 Parameters

#53

Post by romerocruzsa » Thu, Apr 02, 2026, 3:51 AM UTC

endofreach wrote: post #41

sachaa wrote: post #22

Earlier quoted context omitted.

Fair points, especially on GSM8K saturation and Qwen possibly already sitting close to the solution. That said, even if this is mostly "last-mile alignment", the fact that it can be done with such a tiny signal is still interesting, it suggests the gap between capability and behavior might be much smaller (and cheaper to bridge) than we assume.

> the gap between capability and behavior might be much smaller Can you elaborate a bit on what you mean with the gap?

[dead]

Re: TinyLoRA – Learning to Reason in 13 Parameters

#54

Post by AndrewB234 » Thu, Apr 02, 2026, 7:21 PM UTC

[flagged]