Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

101–110 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#101

Post by tim333 »

imiric wrote: post #92
tim333 wrote: post #84

Earlier quoted context omitted.

>There's nothing "omniscient" or "dim-witted" about these tools I disagree in that that seems quite a good way of describing them. All language is a bit inexact. Also I don't buy we are no closer to AI than ten years ago - there seem lots going on. Just because LLMs are limited doesn't mean we can't find or add other algorithms - I mean look at alphaevolve for example https://www.technologyreview.com/2025/05/14/11164…

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

I imagine if you asked the LLM why the wolf can't be close to the goat it would give a reasonable answer. I realise it does it by using permutation of tokens but I think you have to judge intelligence by the results rather than the mechanism otherwise you could argue humans can't be intelligent because they are just a bunch of neurons that find patterns.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#102

Post by Jensson »

Kon5ole wrote: post #94
Jensson wrote: post #88

Earlier quoted context omitted.

But if you need a submarine that can swim as agiley as a fish then we still aren't there yet, fish are far superior to submarines in many ways. So submarines might be faster than fish, but there are so many maneuvers that fish can do that the submarine can't. Its the same with here with thinking. So just like computers are better at humans at multiplying numbers, there are still many things we need human intelligence…

The point here (which is from a quote by Dijkstra) is that if the desired result is achieved (movement through water) it doesn't matter if it happens in a different way than we are used to. So if an LLM generates working code, correct translations, valid points relating to complex matters and so on it doesn't matter if it does so by thinking or by some other mechanism. I think that's an interesting point.

> if the desired result is achieved (movement through water) it doesn't matter if it happens in a different way than we are used to

But the point is that the desired result isn't achieved, we still need humans to think.

So we still need a word for what humans do that is different from what LLM does. If you are saying there is no difference then how do you explain the vast difference in capability between humans and LLM models?

Submarines and swimming is a great metaphor for this, since Submarines clearly doesn't swim and thus have very different abilities in water, its way better in some ways but way worse in other ways. So using that metaphor its clear that LLM "thinking" cannot be described with the same words as human thinking since its so different.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#103

Post by Xmd5a »

thomasahle wrote: post #82

All the environments the test (Tower of Hanoi, Checkers Jumping, River Crossing, Block World) could easily be solved perfectly by any of the LLMs if the authors had allowed it to write code. I don't really see how this is different from "LLMs can't multiply 20 digit numbers"--which btw, most humans can't either. I tried it once (using pen and paper) and consistently made errors somewhere.

>Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents

>In this paper, we introduce a novel framework that addresses these challenges by training a smaller, specialized student RL agent using instructions from an LLM-based teacher agent. By incorporating the guidance from the teacher agent, the student agent can distill the prior knowledge of the LLM into its own model. Consequently, the student agent can be trained with significantly less data. Moreover, through further training with environment feedback, the student agent surpasses the capabilities of its teacher for completing the target task.

https://arxiv.org/abs/2311.13373

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#104

Post by Jensson »

tim333 wrote: post #101
imiric wrote: post #92

Earlier quoted context omitted.

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

I imagine if you asked the LLM why the wolf can't be close to the goat it would give a reasonable answer. I realise it does it by using permutation of tokens but I think you have to judge intelligence by the results rather than the mechanism otherwise you could argue humans can't be intelligent because they are just a bunch of neurons that find patterns.

We have had programs that can give good answers to some hard questions for a very long time now. Watson won jeapordy already 2011, but it still wasn't very good at replacing humans.

So that isn't a good way to judge intelligence, computers are so fast and have so much data that you can make programs to answer just about anything pretty well, LLM is able to do that but more automatic. But it still doesn't automate the logical parts yet, just the lookup of knowledge, we don't know how to train large logic models, just large language models.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#105

Post by Jensson »

danielmarkbruce wrote: post #67
sitkack wrote: post #57

Earlier quoted context omitted.

I think you are right, and that the next step function can be achieved using the models we have, either by scaling the inference, or changing the way inference is done.

People are doing all manner of very sophisticated inferency stuff now - it just tends to be extremely expensive for now and... people are keeping it secret.

If it was good enough to replace people then it wouldn't be too expensive, they would have launched it and replaced a bunch of people and made trillions of dollars by now.

So at best their internal models are still just performance multipliers unless some breakthrough happened very recently, it might be a bigger multiplier but that still keeps humans with jobs etc and thus doesn't revolutionize much.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#106

Post by [deleted] »

curious_cat_163 wrote: post #35

> Rather than standard benchmarks (e.g., math problems), we adopt controllable puzzle environments that let us vary complexity systematically Very clever, I must say. Kudos to folks who made this particular choice. > we identify three performance regimes: (1) low complexity tasks where standard models surprisingly outperform LRMs, (2) medium-complexity tasks where additional thinking in LRMs demonstrates advantage, a…

[deleted]

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#107

Post by Jensson »

rafterydj wrote: post #59
bayindirh wrote: post #29

Earlier quoted context omitted.

As a test, I asked Gemini 2.5 Flash and Gemini 2.5 Pro to decode a single BASE64 string. Flash answered correctly in ~2 seconds, at most. Pro answered very wrongly after thinking and elaborating for ~5 minutes. Flash was also giving a wrong answer for the same string in the past, but it improved. Prompt was the same: "Hey, can you decode $BASE64_string?" I have no further comments.

well that's not a very convincing argument. That's just a failure to recognize when the use of a tool- base64 decoder- is needed, not a reasoning problem at all, right?

Translating to BASE64 is a good test to see how well it works as a language translator without changing things, because its the same skill for an AI model.

If the model changes things it means it didn't really capture the translation patterns for BASE64, so then who knows what it will miss when translating between languages if it can't even do BASE64?

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#108

Post by d4rkn0d3z »

I wrote my first MLP 25 years ago. After repeating some early experiments in machine learning from 20 ywars before that. One of the experiments I repeated was in text to speach. It was amazing to set up training runs and return after seveal hours to listen to my supercomputer babble like a toddler. I literally recall listening and being unable to distinguish the output from my NN from that of a real toddler, I happened to be teaching my neice to read around that same time. And when the NN had gained a large vocabulary such that it could fairly proficiently read aloud, I was convinced that I had found my PHD project and a path to AGI.

Further examination and discussion with more experienced researchers gave me pause. They said that one must have a solution, or a significant new approach toward solving the hard problems associated with a research project for it to be viable, otherwise time (and money) is wasted finding new ways to solve the easy problems.

This is a more general principle that can be applied to most areas of endeavour. When you set about research and development that involves a mix of easy, medium, and hard problems, you must solve the hard problems first otherwise you blow your budget finding new ways to solve the easy problems, which nobody cares about in science.

But "AI" has left the realm of science behind and entered the realm of capitalism where several years of meaningless intellectual gyration without ever solving a hard problem may be quite profitable.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#109

Post by jdmoreira »

Jensson wrote: post #99
jdmoreira wrote: post #97

Earlier quoted context omitted.

No. a huge population of humans did while standing on the shoulders of giants.

Humans aren't giants, they stood on the shoulder of other humans. So for AI to be equivalent they should stand on the shoulders of other AI models.

building for thousands of years with a population size in the range between millions and billions at any given time.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#110

Post by drodgers »

Byamarro wrote: post #78
drodgers wrote: post #56

Earlier quoted context omitted.

I think that commenter was disagreeing with this line: > because omniscient-yet-dim-witted models terminate at "superhumanly assistive" It might be that with dim wits + enough brute force (knowledge, parallelism, trial-and-error, specialisation, speed) models could still substitute for humans and transform the economy in short order.

And we have a good example of a dimwitted, brute-force process creating intelligent designs - evolution.

Also corporations, governments etc. - they're capable of things that none of the individuals could do alone.
Post reply on HN