Reasoning models reason well, until they don't • Hacker News Board

Re: Reasoning models reason well, until they don't

#161

Post by alyxya » Fri, Oct 31, 2025, 4:03 PM UTC

usrbinbash wrote: post #149

alyxya wrote: post #23

The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from som…

> I don't think there's evidence that this issue would persist after continuing to scale models to be larger and doing more RL And how much larger do we need to make the models? 2x? 3x? 10x? 100x? How large do they need to get before scaling-up somehow solves everything? Because: 2x larger, means 2x more memory and compute required. Double the cost or half the capacity. Would people still pay for this tech if it doub…

Scaling applies to multiple dimensions simultaneously over time. A frontier model today could be replicated a year later with a model half the size, with a quarter of the FLOPS, etc. I don’t know the real numbers for optimization scaling, but you could check out NanoGPT speedrun [1] as an example.

The best solution in the meantime is giving the LLM a harness that allows tool use like what coding agents have. I suspect current models are fully capable of solving arbitrary complexity artificial reasoning problems here, provided that they’re used in the context of a coding agent tool.

[1] https://github.com/KellerJordan/modded-nanogpt

Re: Reasoning models reason well, until they don't

#162

Post by squidproquo » Fri, Oct 31, 2025, 4:49 PM UTC

ftalbot wrote: post #49

My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

Every token in a response has an element of randomness to it. This means they’re non-deterministic. Even if you set up something within their training data there is some chance that you could get a nonsense, opposite, and/or dangerous result. The chance of that may be low because of things being set up for it to review its result, but there is no way to make a non-deterministic answer fully bound to solving or reason…

The non-determinism is part of the allure of these systems -- they operate like slot machines in a casino. The dopamine hit of getting an output that appears intelligent and the variable rewards keeps us coming back. We down-weight and ignore the bad outputs. I'm not saying these systems aren't useful to a degree, but one should understand the statistical implications on how we are collectively perceiving their usefulness.

Re: Reasoning models reason well, until they don't

#163

Post by MangoToupe » Fri, Oct 31, 2025, 4:51 PM UTC

sodality2 wrote: post #140

MangoToupe wrote: post #127

Earlier quoted context omitted.

> The theory is that they don't know that there are entities that know things they don't. This seems like a rather awkward way of putting it. They may just lack conceptualization or abstraction, making the above statement meaningless.

The exact title of the capacity is 'theory of mind' - for example, chimpanzees have a limited capacity for it in that they can understand others' intentions, but they seemingly do not understand false beliefs (this is what GP mentioned). https://doi.org/10.1016/j.tics.2008.02.010

Theory of mind is a distinct concept that isn't necessary to explain this behavior. Of course, it may follow naturally, but it strikes me as ham-fisted projection of our own cognition onto others. Ironically, a rather greedy theory of mind!

Re: Reasoning models reason well, until they don't

#164

Post by squidbeak » Fri, Oct 31, 2025, 5:06 PM UTC

weltensturm wrote: post #141

squidbeak wrote: post #120

Earlier quoted context omitted.

This I'm afraid is rubbish. Scientific proofs categorically don't depend on philosophical beliefs. Reality is measurable and the properties measured don't care about philosophy.

> Reality is measurable Heisenberg would disagree.

Are you arguing that the uncertainty principle derives from philosophy rather than math?

Re: Reasoning models reason well, until they don't

#165

Post by squidbeak » Fri, Oct 31, 2025, 5:07 PM UTC

darkwater wrote: post #138

squidbeak wrote: post #135

Earlier quoted context omitted.

> Science is exact without being finished

Being exact doesn't mean it is not an approximation, which was the initial topic. Being exact in science means that 2+2=4 and that can be demonstrated following a logical chain. But that doesn't make our knowledge of the universe exact. It is still an approximation. What it can be "exact" is how we obtain and reproduce the current knowledge we have of it.

The speed of light, or plank's constant - are these approximations?

Re: Reasoning models reason well, until they don't

#166

Post by tekno45 » Fri, Oct 31, 2025, 5:41 PM UTC

tempfile wrote: post #96

hirako2000 wrote: post #51

Has any one ever found an ML/AI paper that make claims that RLMs can reason? When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.

I don't understand what point you are making. Doesn't the name "Reasoning language models" claim that they can reason? Why do you want to see it explicitly written down in a paper?

By renaming this binary to a "Mind reading language model" We now can read your mind and predict your choices just by chatting.

Don't ask how it works cuz its called a "Mind reading language model" duh.

Re: Reasoning models reason well, until they don't

#167

Post by tekno45 » Fri, Oct 31, 2025, 5:42 PM UTC

lingrush4 wrote: post #78

Is that really the best title the authors could come up with? Up next: "Lawn mowers are good at cutting grass until they aren't"

When the news cycle has been "lawnmowers can now do anything, throw away your kitchenaide" its a pretty relevant title.

Re: Reasoning models reason well, until they don't

#168

Post by ares623 » Fri, Oct 31, 2025, 5:55 PM UTC

kerabatsos wrote: post #143

How is that different than human reasoning?

I’d like $500B to just be the way I am thanks.

Re: Reasoning models reason well, until they don't

#169

Post by jeremyjh » Fri, Oct 31, 2025, 6:32 PM UTC

alyxya wrote: post #160

jeremyjh wrote: post #77

Earlier quoted context omitted.

The burden of evidence here is on you. They don’t need to prove LRMs can’t scale to meet these problems; their only claim is current models can’t handle these problems. Others will take this up as a challenge - and chances may be good they will overcome it. This is how science works.

They can’t claim current models aren’t able to handle these problems if they didn’t use a setup similar to coding agents like Claude Code and OpenAI Codex. Using a suboptimal setup is akin to verbally telling a person the whole reasoning problem without letting them write down notes and expecting them to memorize and solve it after only hearing it once.

If the models can’t do it they can make that claim. If you want to make claims about agents then design that experiment, collect the data and write a paper. That is how science works.

Re: Reasoning models reason well, until they don't

#170

Post by rdedev » Fri, Oct 31, 2025, 6:37 PM UTC

alyxya wrote: post #160

jeremyjh wrote: post #77

Earlier quoted context omitted.

The burden of evidence here is on you. They don’t need to prove LRMs can’t scale to meet these problems; their only claim is current models can’t handle these problems. Others will take this up as a challenge - and chances may be good they will overcome it. This is how science works.

They can’t claim current models aren’t able to handle these problems if they didn’t use a setup similar to coding agents like Claude Code and OpenAI Codex. Using a suboptimal setup is akin to verbally telling a person the whole reasoning problem without letting them write down notes and expecting them to memorize and solve it after only hearing it once.

The thing they are testing for is reasoning performance. It makes sense to not give tool access.

This is same as the critiques of the LLM paper by apple where they showed that LLMs fail to solve the tower of hanoi problem after a set number of towers. The test was to see how well these models can reason out a long task. People online were like they could solve that problem if they had access to a coding enviornment. Again the test was to check reasoning capability not if it knew how to code and algorithm to solve the problem.

If model performance degrade a lot after a number of reasoning steps it's good to know where the limits are. Wheather the model had access to tools or not is orthogonal to this problem