Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

221–230 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#221

Post by naijaboiler »

jqpabc123 wrote: post #210
bilsbie wrote: post #207

Earlier quoted context omitted.

In this case it found generic advice and was confusing itself.

That's one explanation. Another could be that it simply has no real *understanding* of anything. It simply did a statistical comparison of the question to the available advice and picked the best match --- kinda what a search engine might do. Expecting *understanding* from a synthetic, statistical process will often end in disappointment.

Yup. It’s time for us to just accept that LLMs are “similar in meaning” machines not “thinking/ understanding” machines

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#222

Post by sroussey »

bilsbie wrote: post #205

I wonders if there’s past symbolic reasoning research we could integrate into LLMs. They’re really good at parsing text and understanding the relationships between objects ie getting the “symbols” correct. Maybe we plug into something like prolog (or other such strategies?)

That’s called nuero-symbolic AI and there are people working on it.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#223

Post by yojo »

jmogly wrote: post #208
piskov wrote: post #202

All "reasoning" models hit a complexity wall where they completely collapse to 0% accuracy. No matter how much computing power you give them, they can't solve harder problems. This research suggests we're not as close to AGI as the hype suggests. Current "reasoning" breakthroughs may be hitting fundamental walls that can't be solved by just adding more data or compute. Apple's researchers used controllable puzzle env…

I think this might be part of the reason Apple is “behind” on generative AI … LLMs have not really proven to be useful outside of relatively niche areas such as coding assistants, legal boiler plate and research, and maybe some data science/analysis which I’m less familiar with Other “end user” facing use cases have so far been comically bad or possibly harmful, and they just don’t meet the quality bar for inclusion…

I have friends that do pretty disparate things (e.g. education consulting, grant writing, solar project planning, etc). They all use LLMs in aspects of their jobs; tasks like rewording emails for tone, extracting themes from brainstorming sessions, rough drafting project plans, etc.

None of them are doing the equivalent of “vibe-coding”, but they use LLMs to get 20-50% done, then take over from there.

Apple likes to deliver products that are polished. Right now the user needs to do the polishing with LLM output. But that doesn’t mean it isn’t useful today

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#224

Post by lukev »

jqpabc123 wrote: post #216
piskov wrote: post #202

All "reasoning" models hit a complexity wall where they completely collapse to 0% accuracy. No matter how much computing power you give them, they can't solve harder problems. This research suggests we're not as close to AGI as the hype suggests. Current "reasoning" breakthroughs may be hitting fundamental walls that can't be solved by just adding more data or compute. Apple's researchers used controllable puzzle env…

No matter how much computing power you give them, they can't solve harder problems. Why would anyone ever expect otherwise? These models are inherently handicapped and always will be in terms of real world experience. They have no real grasp of things people understand intuitively --- like time or money or truth ... or even death. They only *reality* they have to work from is a flawed statistical model built from the…

I agree, obviously, but half the internet is still running around claiming we're on the verge of a singularity, so demonstrating the actual limitations of these systems concretely is important.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#225

Post by halfcat »

bilsbie wrote: post #204

Interestingly I just hit an example of this. Highly specific but I was asking about pickleball strategy and grok and Claude both couldn’t seem to understand you can’t aim at the opponent’s feet when you’re hitting up. Just kept regurgitating internet advice and I couldn’t get it to understand the reasoning on why it was wrong.

It’s almost as if intelligence requires embodiment in a physical world before it can gain tacit understanding and build a model of that environment, and that summarizing the embodiment of others isn’t sufficient.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#226

Post by avsteele »

People are drawing erroneous conclusions from this.

My read of this is that the paper demonstrates that given a particular model (and the problems examined with it) that giving more thought tokens does not help on problems above a certain complexity. It does not say anything about the capabilities of future, larger, models to handle more complex tasks. (NB: humans trend similarly)

My concern is that people are extrapolating from this to conclusions about LLM's generally, and this is not warranted

The only part about this i find even surprising is he abstract's conclusion (1): that 'thinking' can lead to worse outcomes for certain simple problem. (again though, maybe you can say humans are the same here. You can overthink things)

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#227

Post by jqpabc123 »

lukev wrote: post #224
jqpabc123 wrote: post #216

Earlier quoted context omitted.

No matter how much computing power you give them, they can't solve harder problems. Why would anyone ever expect otherwise? These models are inherently handicapped and always will be in terms of real world experience. They have no real grasp of things people understand intuitively --- like time or money or truth ... or even death. They only *reality* they have to work from is a flawed statistical model built from the…

I agree, obviously, but half the internet is still running around claiming we're on the verge of a singularity, so demonstrating the actual limitations of these systems concretely is important.

Half the internet is people with a vested interest in promotion.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#228

Post by rikafurude21 »

Apple, with all the compute and talent anyone could ask for, did not manage to create a frontier LLM model and was forced to bite the bullet and include GPT in their coveted walled garden in order to keep up and not be left behind in our current AI-Hype-(bubble) mania. I suspect this hurts their ego greatly and thats why they feel compelled to "talk trash". Saltiness wont get them anywhere though. I hope they manage to pull through. It would be a shame if they also went where Microsoft currently is.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#229

Post by rafaelero »

imiric wrote: post #92
tim333 wrote: post #84

Earlier quoted context omitted.

>There's nothing "omniscient" or "dim-witted" about these tools I disagree in that that seems quite a good way of describing them. All language is a bit inexact. Also I don't buy we are no closer to AI than ten years ago - there seem lots going on. Just because LLMs are limited doesn't mean we can't find or add other algorithms - I mean look at alphaevolve for example https://www.technologyreview.com/2025/05/14/11164…

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

2025 and we are still discussing if LLM's are intelligent or not, gee.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#230

Post by jqpabc123 »

naijaboiler wrote: post #221
jqpabc123 wrote: post #210

Earlier quoted context omitted.

That's one explanation. Another could be that it simply has no real *understanding* of anything. It simply did a statistical comparison of the question to the available advice and picked the best match --- kinda what a search engine might do. Expecting *understanding* from a synthetic, statistical process will often end in disappointment.

Yup. It’s time for us to just accept that LLMs are “similar in meaning” machines not “thinking/ understanding” machines

If you think about it --- an LLM that could really *grasp* "pickleball" from a text description without ever seeing, playing or "experiencing" the game is not just human level intelligence --- it's superhuman.

And the same applies to a lot of real world situations.

Post reply on HN