Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

141–150 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#141

Post by jackdoe »

dmos62 wrote: post #139
throwaway71271 wrote: post #118

Earlier quoted context omitted.

> And, even the dumber LLMs would slot in naturally into such a process That is what I am struggling with, it is really easy at the moment to slot LLM and make everything worse. Mainly because its output is coming from torch.multinomial with all kinds of speculative decoding and quantizations and etc. But I am convinced it is possible, just not the way I am doing it right now, thats why I am spending most of my time…

What's your approach?

For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers https://punkx.org/jackdoe/30.html and so on.

And of course Yannic Kilcher[4], and also listening in on the paper discussions they do on discord.

Practicing a lot with just doing backpropagation by hand and making toy models by hand to get intuition for the signal flow, and building all kinds of smallish systems, e.g. how far can you push whisper, small qwen3, and kokoro to control your computer with voice?

People think that deepseek/mistral/meta etc are democratizing AI, but its actually Karpathy who teaches us :) so we can understand them and make our own.

[1] https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...

[2] https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PL3FW7Lu3i5...

[3] https://www.youtube.com/@umarjamilai

[4] https://www.youtube.com/@YannicKilcher

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#142

Post by anton-c »

stephc_int13 wrote: post #132

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer. I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition. We're easily fooled by the results of LLM/LRM…

Sounds like we need ai legalese as that's how we navigate the vagueness of language in the real world.

Ofc I imagine they've tried similar things and that it almost takes away the point if u had to prompt that way.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#144

Post by bwfan123 »

thomasahle wrote: post #82

All the environments the test (Tower of Hanoi, Checkers Jumping, River Crossing, Block World) could easily be solved perfectly by any of the LLMs if the authors had allowed it to write code. I don't really see how this is different from "LLMs can't multiply 20 digit numbers"--which btw, most humans can't either. I tried it once (using pen and paper) and consistently made errors somewhere.

> but humans cant do it either

This argument is tired as it keeps getting repeated for any flaws seen in LLMs. And the other tired argument is: wait ! this is a sigmoid curve, and we have not seen the inflection point yet. If someone have me a penny for every comment saying these, I'd be rich by now.

Humans invented machines because they could not do certain things. All the way from simple machines in physics (Archimedes lever) to the modern computer.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#145

Post by kamranjon »

The two interesting things I learned after reading this paper:

Even when given the exact steps needed to arrive at a solution in the prompt, the reasoning models still require just as many steps to reach a workable solution as they would if they weren’t given the solution in the prompt.

The other thing, which seems obvious in hindsight, but I don’t typically use these reasoning models in my day to day - is that it requires a significant amount of tokens to reach the point where reasoning models outperform non-reasoning models by a significant margin.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#146

Post by bwfan123 »

Too wrote: post #72
ivape wrote: post #6

This is easily explained by accepting that there is no such thing as LRMs. LRMs are just LLMs that iterate on its own answers more (or provides itself more context information of a certain type). The reasoning loop on an "LRM" will be equivalent to asking a regular LLM to "refine" its own response, or "consider" additional context of a certain type. There is no such thing as reasoning basically, as it was always a me…

The million dollar question is how far can one get on this trick. Maybe this is exactly how our own brains operate? If not, what fundamental building blocks are missing to get there.

> If not, what fundamental building blocks are missing to get there

If I were to guess, the missing building block is the ability to abstract - which is the ability to create a symbol to represent something. Concrete example of abstraction is seen in the axioms of lambda calculus. 1) ability to posit a variable, 2) ability to define a function using said variable, and 3) the ability to apply functions to things. Abstraction arises from a process in the brain which we have not understood yet and could be outside of computation as we know it per [1]

[1] https://www.amazon.com/Emperors-New-Mind-Concerning-Computer...

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#147

Post by hackinthebochs »

imiric wrote: post #92
tim333 wrote: post #84

Earlier quoted context omitted.

>There's nothing "omniscient" or "dim-witted" about these tools I disagree in that that seems quite a good way of describing them. All language is a bit inexact. Also I don't buy we are no closer to AI than ten years ago - there seem lots going on. Just because LLMs are limited doesn't mean we can't find or add other algorithms - I mean look at alphaevolve for example https://www.technologyreview.com/2025/05/14/11164…

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

>The machine has no semantic understanding of what the data represents.

How do you define "semantic understanding" in a way that doesn't ultimately boil down to saying they don't have phenomenal consciousness? Any functional concept of semantic understanding is captured to some degree by LLMs.

Typically when we attribute understanding to some entity, we recognize some substantial abilities in the entity in relation to that which is being understood. Specifically, the subject recognizes relevant entities and their relationships, various causal dependences, and so on. This ability goes beyond rote memorization, it has a counterfactual quality in that the subject can infer facts or descriptions in different but related cases beyond the subject's explicit knowledge. But LLMs excel at this.

>feed it more tricks so that it doesn't fumble with basic questions like how many "R"s are in "strawberry"

This failure mode has nothing to do with LLMs lacking intelligence and everything to do with how tokens are represented. They do not see individual characters, but sub-word chunks. It's like expecting a human to count the pixels in an image it sees on a computer screen. While not impossible, it's unnatural to how we process images and therefore error-prone.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#148

Post by mjburgess »

thomasahle wrote: post #82

All the environments the test (Tower of Hanoi, Checkers Jumping, River Crossing, Block World) could easily be solved perfectly by any of the LLMs if the authors had allowed it to write code. I don't really see how this is different from "LLMs can't multiply 20 digit numbers"--which btw, most humans can't either. I tried it once (using pen and paper) and consistently made errors somewhere.

The goal isnt to assess the LLM capability at solving any of those problems. The point isnt how good they are at block world puzzles.

The point is to construct non-circular ways of quantifying model performance in reasoning. That the LLM has access to prior exemplars of any given problem is exactly the issue in establishing performance in reasoning, over historical synthesis.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#149

Post by swat535 »

tim333 wrote: post #101
imiric wrote: post #92

Earlier quoted context omitted.

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

I imagine if you asked the LLM why the wolf can't be close to the goat it would give a reasonable answer. I realise it does it by using permutation of tokens but I think you have to judge intelligence by the results rather than the mechanism otherwise you could argue humans can't be intelligent because they are just a bunch of neurons that find patterns.

> you have to judge intelligence by the results rather than the mechanism

This would be the exact opposite conclusion of the Chinese room: https://en.wikipedia.org/wiki/Chinese_room

I think you'd need to offer a stronger counter argument than the one you presented here.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#150

Post by jplusequalt »

Kon5ole wrote: post #94
Jensson wrote: post #88

Earlier quoted context omitted.

But if you need a submarine that can swim as agiley as a fish then we still aren't there yet, fish are far superior to submarines in many ways. So submarines might be faster than fish, but there are so many maneuvers that fish can do that the submarine can't. Its the same with here with thinking. So just like computers are better at humans at multiplying numbers, there are still many things we need human intelligence…

The point here (which is from a quote by Dijkstra) is that if the desired result is achieved (movement through water) it doesn't matter if it happens in a different way than we are used to. So if an LLM generates working code, correct translations, valid points relating to complex matters and so on it doesn't matter if it does so by thinking or by some other mechanism. I think that's an interesting point.

>So if an LLM generates working code

It matters when code bases become hard to parse because the engineers throwing shit together with Cursor have made an ungrokkable ball of shit.

Post reply on HN