LIMO: Less Is More for Reasoning
arxiv.org
LIMO: Less Is More for Reasoning
1–10 of 137 posts
Re: LIMO: Less Is More for Reasoning
#2 To see a World in a Grain of Sand
And a Heaven in a Wild Flower,
Hold Infinity in the palm of your hand
And Eternity in an hour.Re: LIMO: Less Is More for Reasoning
#3In the same way that image diffusion models showed that convincing approximations of the entire visual world could be summarized in a 5GB model, are "reasoning patterns" similarly compressible? Are there actually countably few reasoning patterns that are used across all domains, and as such can be captured with relatively small training sets?
Re: LIMO: Less Is More for Reasoning
#4where's chatbotAI-zero? in the way alpha-go-zero was the best after training with itself? (and only with itself)
Re: LIMO: Less Is More for Reasoning
#5where's chatbotAI-zero? in the way alpha-go-zero was the best after training with itself? (and only with itself)
You don't want that as a product, in the sense that having an AI model train itself by simply having internal conversations without ever looking at any human-written content, might result in something that humans cannot comprehend.
Also, well - there's the technicality of "you don't 'win' a conversation like you can 'win' at Go", so how would you know to reward the model as you're training it?
Re: LIMO: Less Is More for Reasoning
#6I think I've recently read two seemingly contradicting things:
1- LLMs can never generalize theorem proving
2- this paper: "This suggests that contemporary LLMs may already possess rich mathematical knowledge in their parameter space, transforming the challenge from knowledge acquisition to knowledge elicitation"
Not sure what is what anymore!
Re: LIMO: Less Is More for Reasoning
#7where's chatbotAI-zero? in the way alpha-go-zero was the best after training with itself? (and only with itself)
The advantage of alpha-go-zero is that it is constrained to the language of go. If you made two LLM train only off each other they would develop their own language. Maybe they'd be great at reasoning, but we wouldn't understand them. Even humans in that situation would develop jargon, and as time goes on a dialect or language of their own. And humans are a lot more grounded in their language than LLMs.
Re: LIMO: Less Is More for Reasoning
#8where's chatbotAI-zero? in the way alpha-go-zero was the best after training with itself? (and only with itself)
You don't want that as a product, in the sense that having an AI model train itself by simply having internal conversations without ever looking at any human-written content, might result in something that humans cannot comprehend. Also, well - there's the technicality of "you don't 'win' a conversation like you can 'win' at Go", so how would you know to reward the model as you're training it?
Also, well - there's the technicality of "you don't 'win' a conversation like you can 'win' at Go", so how would you know to reward the model as you're training it?
https://i.imgur.com/CBmMSqO.png, perhaps
Re: LIMO: Less Is More for Reasoning
#9I think I've recently read two seemingly contradicting things: 1- LLMs can never generalize theorem proving 2- this paper: "This suggests that contemporary LLMs may already possess rich mathematical knowledge in their parameter space, transforming the challenge from knowledge acquisition to knowledge elicitation" Not sure what is what anymore!
Re: LIMO: Less Is More for Reasoning
#10To see a World in a Grain of Sand And a Heaven in a Wild Flower, Hold Infinity in the palm of your hand And Eternity in an hour.
Come in under the shadow of this impure rock
And I will show you something different from either
Your shadow at morning striding behind you
Or your shadow at evening rising to meet you;
I will show you wisdom in a handful of sand.