Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reasoning models reason well, until they don't

arxiv.org

Post reply on HN
Print view View original

41–50 of 224 posts

Re: Reasoning models reason well, until they don't

#41

Post by fsloth »

cryptonym wrote: post #35
hansmayer wrote: post #26

Earlier quoted context omitted.

Ah yes then, let me then unchain my LLM on those nasty unsolved math and logic problems I've absolutely not be struggling with in the course of my career.

That's the real deal. They say LLM are PhD-level. Despite billion dollars, PhD-LLMs sure are not contributing a lot solving known problems. Except of course few limited marketing stunts.

IMHO that's the key differentiator.

You can give a human PhD an _unsolved problem_ in field adjacent to their expertise and expect some reasonable resolution. LLM PhD:s solve only known problems.

That said humans can also be really bad problem solvers.

If you don't care about solving the problem and only want to create paperwork for bureaucracy I guess you don't care either way ("My team's on it!") but companies that don't go out of business generally recognize pretty soon lack of outcomes where it matters.

Re: Reasoning models reason well, until they don't

#42

Post by sothatsit »

js8 wrote: post #34
sothatsit wrote: post #6

Earlier quoted context omitted.

What do you mean by reasoning? If you mean solving logic problems, then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. Reasoning LLMs can also complete problems like multiplying large numbers, which requires applying some sort of algorithm where the results cannot just be memorised. They also do this much better than standard pre-trained LLMs with no RL. So, that makes m…

> So, that makes me come back to this question of what definition of reasoning do people use that reasoning models do not meet? The models can learn reasoning rules, but they are not able to apply them consistently or recognize the rules they have learned are inconsistent. (See also my other comment which references comments I made earlier.) And I think they can't without a tradeoff, as I commented https://news.ycomb…

Yes, so I think in this case we use different definitions of reasoning. You include reliability as a part of reasoning, whereas I do not.

I would argue that humans are not 100% reliable in their reasoning, and yet we still claim that they can reason. So, even though I would agree that the reasoning of LLMs is much less reliable, careful, and thoughtful than smart humans, that does not mean that they are not reasoning. Rather, it means that their reasoning is more unreliable and less well-applied than people. But they are still performing reasoning tasks (even if their application of reasoning can be flawed).

Maybe the problem is that I am holding out a minimum bar for LLMs to jump to count as reasoning (demonstrated application of logical algorithms to solve novel problems in any domain), whereas other people are holding the bar higher (consistent and logical application of rules in all/most domains).

Re: Reasoning models reason well, until they don't

#43

Post by ukuina »

carlmr wrote: post #32
dspillett wrote: post #16

Earlier quoted context omitted.

Some would consider that to be failing catastrophically. The task is certainly failed.

Halting is sometimes preferable to thrashing around and running in circles. I feel like if LLMs "knew" when they're out of their depth, they could be much more useful. The question is whether knowing when to stop can be meaningfully learned from examples with RL. From all we've seen the hallucination problem and this stopping problem all boil down to this problem that you could teach the model to say "I don't know" b…

> if LLMs "knew" when they're out of their depth, they could be much more useful.

I used to think this, but no longer sure.

Large-scale tasks just grind to a halt with more modern LLMs because of this perception of impassable complexity.

And it's not that they need extensive planning, the LLM knows what needs to be done (it'll even tell you!), it's just more work than will fit within a "session" (arbitrary) and so it would rather refuse than get started.

So you're now looking at TODOs, and hierarchical plans, and all this unnecessary pre-work even when the task scales horizontally very well (if it just jumped into it).

Re: Reasoning models reason well, until they don't

#44

Post by arethuza »

moffkalast wrote: post #24
AlecSchueler wrote: post #17

Earlier quoted context omitted.

I also fail catastrophically when trying to push nails through walls by I expect my hammer to do better.

I have one hammer and I expect it to work on every nail and screw. If it's not a general hammer, what good is it now?

You don't need a "general hammer" - they are old fashioned - you need a "general-purpose tool-building factory factory factory":

https://www.danstroot.com/posts/2018-10-03-hammer-factories

Re: Reasoning models reason well, until they don't

#45

Post by My_Name »

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it.

The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confidence.

Re: Reasoning models reason well, until they don't

#46

Post by nakamoto_damacy »

LLMs falter because likelihood-driven pattern completion doesn’t enforce coherence across uncertainty (probability), representation (geometry), composition (category), and search (reasoning). To get robust reasoning, we need these layers to be explicit, typed, and mutually constraining—with verification and calibrated belief updates in the loop.

I was interviewed about this recently, and mentioned the great work of a professor of CS and Law who has been building the foundations for this approach. My own article about it was recently un-linked due to a Notion mishap (but available if anyone is interested - I have to publish it again)

https://www.forbes.com/sites/hessiejones/2025/09/30/llms-are...

Re: Reasoning models reason well, until they don't

#47

Post by hansmayer »

cryptonym wrote: post #35
hansmayer wrote: post #26

Earlier quoted context omitted.

Ah yes then, let me then unchain my LLM on those nasty unsolved math and logic problems I've absolutely not be struggling with in the course of my career.

That's the real deal. They say LLM are PhD-level. Despite billion dollars, PhD-LLMs sure are not contributing a lot solving known problems. Except of course few limited marketing stunts.

I wish our press was not effectively muted or bought by the money, so none of the journos has cojones to call out the specific people who were blabbing about PhD-levels, AGI etc. They should be god damn calling them out every single day, essentially doing their job, but they are now too timid for that.

Re: Reasoning models reason well, until they don't

#49

Post by ftalbot »

My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

Every token in a response has an element of randomness to it. This means they’re non-deterministic. Even if you set up something within their training data there is some chance that you could get a nonsense, opposite, and/or dangerous result. The chance of that may be low because of things being set up for it to review its result, but there is no way to make a non-deterministic answer fully bound to solving or reasoning anything assuredly, given enough iterations. It is designed to be imperfect.

Re: Reasoning models reason well, until they don't

#50

Post by vidarh »

ffsm8 wrote: post #40
antonvs wrote: post #20

Earlier quoted context omitted.

I’ve had models “redirect the problem to someone who has a greater likelihood of not failing”. Gemini in particular will do this when it runs into trouble. I don’t find all these claims that models are somehow worse than humans in such areas convincing. Yes, they’re worse in some respects. But when you’re talking about things related to failures and accuracy, they’re mostly superhuman. For example, how many humans ca…

> For example, how many humans can write hundred of lines of code (in seconds mind you) and regularly not have any syntax errors or bugs? Ez, just use codegen. Also the second part (not having bugs) is unlikely to be true for the LLM generated code, whereas traditional codegen will actually generate code with pretty much no bugs.

I have Claude reducing the number of bugs in my traditional codegen right now.
Post reply on HN