The Illusion of Thinking: Strengths and limitations of reasoning models [pdf] • Hacker News Board

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#171

Post by farts_mckensy » Sat, Jun 07, 2025, 8:18 PM UTC

visarga wrote: post #7

farts_mckensy wrote: post #3

[flagged]

> when you drill down to first principles It is a trap to consider that first principles perspective is sufficient. Like, if I tell you before 1980: "iterate over f(z) = z^2 + c" there is no way you are going to guess fractals emerge. Same with the rules for Conway's Game of Life - seeing the code you won't guess it makes gliders and guns. My point is that recursion creates its own inner opacity, it is irreducible, s…

What with this dingus coming at me with recursion? Hey, that's my move, bro. Seriously though, everything you said is totally unnecessary. All I'm saying is, in order to refute the claim that LLMs can't reason, you ought to have your own coherent definition of the word in order to falsify the claim. Recursion is a coherent notion. If you want to include that in the definition, have at it. That actually supports my broader view of reasoning.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#172

Post by farts_mckensy » Sat, Jun 07, 2025, 8:21 PM UTC

Terr_ wrote: post #23

farts_mckensy wrote: post #11

Earlier quoted context omitted.

Reasoning exists on a spectrum, not as a binary property. I'm not claiming that LLMs reason identically to humans in all contexts. You act as if statistical processes can’t ever scale into reasoning, despite the fact that humans themselves are gradient-trained statistical learners over evolutionary and developmental timescales.

> cretins proclaiming that LLMs aren't truly capable of reasoning > Reasoning is not difficult to define > Reasoning exists on a spectrum > statistical processes [can] scale into reasoning It seems like quite a descent here, starting with the lofty heights of condemning skeptics as "cretins" and insisting the definition is easy... down to what sounds like the introduction to a flavor of panpsychism [0], where even wa…

You think because reasoning exists on a spectrum, therefore everything is conscious? You sound a bit too desperate to try to prove me wrong. Also you edited your comment post hoc to add sources. Who cares?

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#173

Post by ngneer » Sat, Jun 07, 2025, 8:33 PM UTC

dist-epoch wrote: post #158

ngneer wrote: post #153

Earlier quoted context omitted.

I do not know whether the state of the art is able to reason or not. The textbook example you gave is admittedly not very interesting. What you are hearing from people is that parroting is not reasoning, which is true. I wonder if the state of the art can reason its way through the following: "Adam can count to 14000. Can Adam count to 13500?" The response needs to be affirmative for every X1 and X2 such that X2 The…

> "Adam can lift 1000 pounds of steel. Can Adam lift 1000 pounds of feathers?" Worked for me: https://chatgpt.com/share/6844813a-6e4c-8006-b560-c0be223eeb... gemma3-27b, a small model, had an interesting take: > This is a classic trick question! > While Adam can lift 1000 pounds, no, he likely cannot lift 1000 pounds of feathers. > Volume: Feathers take up a huge amount of space for their weight. 1000 pounds of feath…

Thanks for trying these out :). Highlights the often subtle difference between knowing the answer and deducing the answer. Feathers could be ground into a pulp and condensed, too. I am not trying to be clever, just seems like the response is a canned answer.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#174

Post by Kon5ole » Sat, Jun 07, 2025, 9:22 PM UTC

Jensson wrote: post #102

Kon5ole wrote: post #94

Earlier quoted context omitted.

The point here (which is from a quote by Dijkstra) is that if the desired result is achieved (movement through water) it doesn't matter if it happens in a different way than we are used to. So if an LLM generates working code, correct translations, valid points relating to complex matters and so on it doesn't matter if it does so by thinking or by some other mechanism. I think that's an interesting point.

> if the desired result is achieved (movement through water) it doesn't matter if it happens in a different way than we are used to But the point is that the desired result isn't achieved, we still need humans to think. So we still need a word for what humans do that is different from what LLM does. If you are saying there is no difference then how do you explain the vast difference in capability between humans and L…

>If you are saying there is no difference then how do you explain the vast difference in capability between humans and LLM models?

No I completely agree that they are different, like swimming and propulsion by propellers - my point is that the difference may be irrelevant in many cases.

Humans haven't been able to beat computers in chess since the 90s, long before LLM's became a thing. Chess engines from the 90s were not at all "thinking" in any sense of the word.

It turns out "thinking" is not required in order to win chess games. Whatever mechanism a chess engine uses gets better results than a thinking human does, so if you want to win a chess game, you bring a computer, not a human.

What if that also applies to other things, like translation of languages, summarizing complex texts, writing advanced algorithms, realizing implications from a bunch of seemingly unrelated scientific papers, and so on. Does it matter that there was no "thinking" going on, if it works?

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#175

Post by farts_mckensy » Sat, Jun 07, 2025, 9:47 PM UTC

geraneum wrote: post #131

farts_mckensy wrote: post #5

Earlier quoted context omitted.

Your god analogy is clumsy in this case. We aren't talking about something fantastical here. Reasoning is not difficult to define. We can go down that road if you'd like. Rather, the problem is, once you do define it, you will quickly find that LLMs are capable of it. And that makes human exceptionalists a bit uncomfortable.

> Rather, the problem is, once you do define it, you will quickly find that LLMs are capable of it. That’s really not what’s happening though. People who claim LLMs can do X and Y often don’t even understand how LLMs work. The opposite is also true. They just open a prompt and get an output and shout Eureka. Of course not everyone is like this, but majority are. It’s similar to what we think about thinking itself. Yo…

Notice how nearly every comment is just dancing around the issue and nitpicking instead of just owning up to whether or not they think LLMs are capable of reasoning.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#176

Post by farts_mckensy » Sat, Jun 07, 2025, 9:51 PM UTC

earnestinger wrote: post #18

farts_mckensy wrote: post #5

Earlier quoted context omitted.

Your god analogy is clumsy in this case. We aren't talking about something fantastical here. Reasoning is not difficult to define. We can go down that road if you'd like. Rather, the problem is, once you do define it, you will quickly find that LLMs are capable of it. And that makes human exceptionalists a bit uncomfortable.

Hypothetically. Imagine a simple, but humongous hash table, that maps every possible prompt of 20000 letters to the most appropriate answer in current time. (How? Let’s say outsourcing to east asia or aliens…) Would you say such mechanism is doing reasoning?

Let's outsource it to your mothers house instead.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#177

Post by stephc_int13 » Sat, Jun 07, 2025, 9:59 PM UTC

wslh wrote: post #151

stephc_int13 wrote: post #132

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer. I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition. We're easily fooled by the results of LLM/LRM…

Human language is more powerful than its surface syntax or semantics: it carries meaning beyond formal correctness. We often communicate effectively even with grammatically broken sentences, using jokes, metaphors, or emotionally charged expressions. This richness makes language a uniquely human cognitive layer, shaped by context, culture, and shared experience. While it's not foundational in the same way as sensorim…

I agree that language is even more useful as a cognitive tool than as a communication medium.

But that is not my point. The map is not the territory, and this map (language) is too poor to build something that is going to give more than what it was fed with.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#178

Post by IshKebab » Sat, Jun 07, 2025, 10:05 PM UTC

actinium226 wrote: post #24

Man, remember when everyone was like 'AGI just around the corner!' Funny how well the Gartner hype cycle captures these sorts of things

Yeah it's already been 2½ years! How long does it take to develop artificial life anyway? Surely no more than 3 years? I demand my money back!

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#179

Post by mrbungie » Sat, Jun 07, 2025, 10:09 PM UTC

antithesizer wrote: post #165

stephc_int13 wrote: post #132

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer. I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition. We're easily fooled by the results of LLM/LRM…

Language mediates those interactions with the world. There is no unmediated interaction with the world. Those moments when one feels most directly in contact with reality, that is when one is so deep down inside language that one cannot see daylight at all.

I don't know about you, but as far as I can tell I mediate and manipulate the world with my body and senses without necessarily using language. In fact, I can often do both at once, for example, thinking about something entirely unrelated while jogging, and still making physical decisions and actions without invoking language at all. Plus, animals (especially lower order like amoebas) also mediate with the world without needing language.

As far as we can tell without messing with complex experiental concepts like qualia and the possibility of philosophical zombies, language mainly helps higher order animals communicate with other animals and (maybe) keep a train of thought, though there are records of people that say that they don't. And now also it allows humans talk to LLMs.

But I digress, I would say this is an open academic debate. Suggesting that there is always language deep down is speculation.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#180

Post by mrbungie » Sat, Jun 07, 2025, 10:39 PM UTC

thomasahle wrote: post #157

bwfan123 wrote: post #144

Earlier quoted context omitted.

> but humans cant do it either This argument is tired as it keeps getting repeated for any flaws seen in LLMs. And the other tired argument is: wait ! this is a sigmoid curve, and we have not seen the inflection point yet. If someone have me a penny for every comment saying these, I'd be rich by now. Humans invented machines because they could not do certain things. All the way from simple machines in physics (Archim…

> Humans invented machines because they could not do certain things. If your disappointment is that the LLM didn't invent a computer to solve the problem, maybe you need to give it access to physical tools, robots, labs etc.

Nah, even if we follow such a weak "argument" the fact is that, ironically, the evidence shown in this and other papers point towards the idea that even if LRMs did have access to physical tools, robots labs, etc*, they probably would not be able to harness them properly. So even if we had an API-first world (i.e. every object and subject in the world can be mediated via a MCP server), they wouldn't be able to perform as well as we hope.

Sure, humans may fail doing a 20 digit multiplication problems but I don't think that's relevant. Most aligned, educated and well incentivized humans (such as the ones building and handling labs) will follow complex and probably ill-defined instructions correctly and predictably, instructions harder to follow and interpret than an exact Towers of Hanoi solving algorithm. Don't misinterpret me, human errors do happen in those contexts because, well, we're talking about humans, but not as catastrophically as the errors committed by LRMs in this paper.

I'm kind of tired of people comparing humans to machines in such simple and dishonest ways. Such thoughts pollute the AI field.

*In this case for some of the problems the LRMs were given an exact algorithm to follow, and they didn't. I wouldn't keep my hopes up for an LRM handling a full physical laboratory/factory.