While there're interesting findings here, https://arxiv.org/pdf/2502.03373 (also with a lot of good findings) suggested some contradicting theory on the critical mass of training process/data for the sake of reasoning capability.
LIMO: Less Is More for Reasoning
31–40 of 137 posts
Re: LIMO: Less Is More for Reasoning
#32My conclusion from all that I'm reading lately is that LLMs cannot do deduction but they can fake it real good.
I mean, you wouldn't use this brand of AI to plot your path to Mars. Well, you could, BUT you'll also want to validate the path or risk dying.
But this AI is good enough for Elon and his ilk. Because Elon's not gonna get into the capsule, you are.
Because you are not the master of this AI, you are the validator.
Re: LIMO: Less Is More for Reasoning
#33Cool result, but worth highlighting two points:
- Model is finetuned from Qwen-2.5 Instruct, which includes millions of specially filtered math examples in both pretraining and supervised fine-tuning already.
- To generate the perfect 817 math examples for LIMO, they used state of the art models like R1 to filter down from an initial pool of 10 million math problems. In other words, a whole lot of intelligence was used to craft a maximally informative and distilled set of fine-tuning data. It’s not very clear to me if this is more or less impressive than getting the same result by simply fine-tuning on the 10 million initial pool, but I suppose that would make for a worse headline.
Re: LIMO: Less Is More for Reasoning
#34My guess at the upshot: Some domains, like math, are general but have outsized effective vocabularies like all possible numbers, which makes them more expensive to train by the same method that works for domains of regular-sized vocabularies. If you train for reasoning steps in such a problem domain, you can reinforce the comparatively few general terms of the vocabulary like "add", "inverse", "solve". And that leave…
A connected question -- has there been an LLM that is a perfect calculator ? I.e. you give it a expression involving standard operations +/- and (say) integer numbers, standard operations and it should returns always a correct result.
I don't remember seeing any papers on this (but i'm not an expert)
Re: LIMO: Less Is More for Reasoning
#35Earlier quoted context omitted.
I think the way to swallow this bitter pill is to acknowledge they can "generalize" because all human knowledge is actually a relatively "small" finite distribution that models are now big enough to pattern match on.
Calling human knowledge small is hyperbole. I cannot get any LLM even close to giving accurate answers related to the things I know. They simply do not know what I, a single human being, knows. That's simply because I'm a subject matter expert on somewhat niche topics. There are easily hundreds of thousands of people like me out there. There's simply no way an LLM can even train on all of that because each bit of tru…
[deleted]
Re: LIMO: Less Is More for Reasoning
#36Love prepending 'explain' to arxiv links these days xD https://explainarxiv.org/abs/2502.03387
Re: LIMO: Less Is More for Reasoning
#37Earlier quoted context omitted.
You don't want that as a product, in the sense that having an AI model train itself by simply having internal conversations without ever looking at any human-written content, might result in something that humans cannot comprehend. Also, well - there's the technicality of "you don't 'win' a conversation like you can 'win' at Go", so how would you know to reward the model as you're training it?
I do... I want a chatbot that can automatically magic up proofs that all my code is correct for instance. I don't care if I understand the proofs. I care if some tool that checks proofs understands them, and that's a mechanical game just like go or chess.
I think that there is a strong limit to that: if you don't understand the proofs, you're going to have a hard time understanding when the model explains to you why your code is not correct.
Re: LIMO: Less Is More for Reasoning
#38Cool result, but worth highlighting two points: - Model is finetuned from Qwen-2.5 Instruct, which includes millions of specially filtered math examples in both pretraining and supervised fine-tuning already. - To generate the perfect 817 math examples for LIMO, they used state of the art models like R1 to filter down from an initial pool of 10 million math problems. In other words, a whole lot of intelligence was us…
Yeah, but it's cheaper.
The context right now is that OpenAI, with first-mover advantage, cutting-edge-hardware, and tens of billions of dollars of investment, are not getting benchmark performance better than Chinese-developed models that are trained with cut-down nvidia GPUs and a lot less money.
Re: LIMO: Less Is More for Reasoning
#39Love prepending 'explain' to arxiv links these days xD https://explainarxiv.org/abs/2502.03387
I don’t have enough karma to downvote adverts.
Re: LIMO: Less Is More for Reasoning
#40where's chatbotAI-zero? in the way alpha-go-zero was the best after training with itself? (and only with itself)
We kind-of have that in DeepSeek-R1-zero [1], but it has problem. From the original authors: > With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. A lot of these we can probably solve, but as other have pointed out we want a model that humans can conver…
Despite the similar "zero" names, DeepSeek-R1 Zero and AlphaGo Zero have nothing in common.
AlphaGo came before AlphaGo Zero; it was trained on human games, then improved further via self-play. The later AlphaGo Zero proved that pre-training on human games was not necessary, and the model could learn from scratch (i.e. from zero) just via self-play.
For DeepSeek-R1, or any reasoning model, training data is necessary, but hard to come by. One of the main contributions of the DeepSeek-R1 paper was describing their "bootstrapping" (my term) process whereby they started with a non-reasoning model, DeepSeek-V3, and used a three step process to generate more and more reasoning data from that (+ a few other sources) until they had enough to train DeepSeek-R1, which they then further improved with RL.
DeepSeek-R1 Zero isn't a self-play version of DeepSeek-R1 - it was just the result of the first (0th) step of this bootstrapping process whereby they used RL to finetune DeepSeek-V3 into the (somewhat of an idiot savant - one trick pony) R1 Zero model that was then capable of generating training data for the next bootstrapping step.