Has any one ever found an ML/AI paper that make claims that RLMs can reason?
When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.
Reasoning models reason well, until they don't
51–60 of 224 posts
Re: Reasoning models reason well, until they don't
#52Earlier quoted context omitted.
Do you? Don't you just halt and say this is too complex?
Some would consider that to be failing catastrophically. The task is certainly failed.
This seems to be the stance of creators of agentic coders. They are so bound on creating something, even if this something makes no sense whatsoever.
Re: Reasoning models reason well, until they don't
#53LLMs are like students, they can reason a bit, but real understanding still takes time and practice.
What? The LLMs are nothing like students (or any other human for that matter).
Re: Reasoning models reason well, until they don't
#54LLMs falter because likelihood-driven pattern completion doesn’t enforce coherence across uncertainty (probability), representation (geometry), composition (category), and search (reasoning). To get robust reasoning, we need these layers to be explicit, typed, and mutually constraining—with verification and calibrated belief updates in the loop. I was interviewed about this recently, and mentioned the great work of a…
Richard Sutton's interview on Dwarkesh's podcast hit at this same point. The implicit world models in LLMs are insufficient.
Re: Reasoning models reason well, until they don't
#55But I also fail catastrophically once a reasoning problem exceeds modest complexity.
Do you? Don't you just halt and say this is too complex?
Ah yes, the function that halts if the input problem would take too long to halt.
But yes, I assume you mean they abort their loop after a while, which they do.
This whole idea of a "reasoning benchmark" doesn't sit well with me. It seems still not well-defined to me.
Maybe it's just bias I have or my own lack of intelligence, but it seems to me that using language models for "reasoning" is still more or less a gimmick and convenience feature (to automate re-prompts, clarifications etc, as far as possible).
But reading this pop-sci article from summer 2022 seems like this definition problem hasn't changed very much since then.
Although it's about AI progress before ChatGPT and it doesn't even mention the GPT base models. Sure, some of the tasks mentioned in the article seem dated today.
But IMO, there is still no AI model that can be trusted to, for example, accurately summarize a Wikipedia article.
Not all humans can do that either, sure. But humans are better at knowing what they don't know, and deciding what other humans can be trusted. And of course, none of this is an arithmetic or calculation task.
https://www.science.org/content/article/computers-ace-iq-tes...
Re: Reasoning models reason well, until they don't
#56I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
If by task you mean the written, intellectual variety, maybe.
Re: Reasoning models reason well, until they don't
#57I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…
I think a good test of this seems to be to provide an image and get the model to predict what will happen next/if x occurs. They fail spectacularly at Rube-Goldberg machines. I think developing some sort of dedicated prediction model would help massively in extrapolating data. The human subconscious is filled with all sorts of parabolic prediction, gravity, momentum and various other fast-thinking paths that embed these calculations.
Re: Reasoning models reason well, until they don't
#58> [...] recent studies show that transformers and LLMs fail catastrophically once reasoning problems exceed modest complexity. We revisit these findings through the lens of large reasoning models (LRMs) -- LLMs fine-tuned with incentives for step-by-step argumentation and self-verification This was the obvious outcome of the study (don't get me wrong, obvious outcomes are still worth having research on). "LRMs" *are*…
Don't they have a significant RL component? The "we'll just make it bigger" idea that was peddled a lot after GPT3.5 was nonsense, but that's not the only thing they're doing right now.
"We'll just make it bigger" works. RLVR just gives better performance gains and spends less inference compute - as long as you have a solid way of verifying the tasks.
A simplified way of thinking about it is: pretraining gives LLMs useful features, SFT arranges them into useful configurations, RLVR glues them together and makes them work together well, especially in long reasoning traces. Makes sense to combine it all in practice.
How much pretraining gives an LLM depends on the scale of that LLM, among other things. But raw scale is bounded by the hardware capabilities and the economics - of training and especially of inference.
Scale is still quite desirable - GPT-4.5 scale models are going to become the norm for high end LLMs quite soon.
Re: Reasoning models reason well, until they don't
#59I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
[deleted]
Re: Reasoning models reason well, until they don't
#60I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…
Every token in a response has an element of randomness to it. This means they’re non-deterministic. Even if you set up something within their training data there is some chance that you could get a nonsense, opposite, and/or dangerous result. The chance of that may be low because of things being set up for it to review its result, but there is no way to make a non-deterministic answer fully bound to solving or reason…
You are making a wrong leap from non-deterministic process to uncontrollable result. Most of the parallel algorithms are non-deterministic. There might be no guarantee about the order of calculation or even sometimes the final absolute result. However, even when producing different final results, the algorithm can still guarantee characteristics about the result.
The hard problem then is not to eliminate non-deterministic behavior, but find a way to control it so that it produces what you want.