Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

201–210 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#201

Post by bayindirh »

anonzzzies wrote: post #199
bayindirh wrote: post #93

Earlier quoted context omitted.

Waymo is a popular argument in self-driving cars, and they do well. However, Waymo is Deep Blue of self-driving cars. Doing very well in a closed space . As a result of this geofencing, they have effectively exhausted their search space, hence they work well as a consequence of lack of surprises. AI works well when search space is limited, but General AI in any category needs to handle a vastly larger search space, a…

Yeah, AI has been good for a long time in limited search space areas. So good that many of these things that were called AI in the past are not called AI now, but 'just' 'algorithm'.

Everything is “just” an algorithm. LLM is a weighted graph with some randomization which is tuned with tons of data. You have input and output encoders on top of it.

That’s all.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#202

Post by piskov »

All "reasoning" models hit a complexity wall where they completely collapse to 0% accuracy.

No matter how much computing power you give them, they can't solve harder problems.

This research suggests we're not as close to AGI as the hype suggests.

Current "reasoning" breakthroughs may be hitting fundamental walls that can't be solved by just adding more data or compute.

Apple's researchers used controllable puzzle environments specifically because:

• They avoid data contamination • They require pure logical reasoning • They can scale complexity precisely • They reveal where models actually break

Models could handle 100+ moves in Tower of Hanoi puzzles but failed after just 4 moves in River Crossing puzzles.

This suggests they memorized Tower of Hanoi solutions during training but can't actually reason.

https://x.com/RubenHssd/status/1931389580105925115

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#203

Post by intended »

Kon5ole wrote: post #83
imiric wrote: post #71

Earlier quoted context omitted.

> I think the intuition the authors are trying to capture is that they believe the models are omniscient, but also dim-witted. We keep assigning adjectives to this technology that anthropomorphize the neat tricks we've invented. There's nothing "omniscient" or "dim-witted" about these tools. They have no wit. They do not think or reason. All Large "Reasoning" Models do is generate data that they use as context to gen…

>They have no wit. They do not think or reason. Computers can't think and submarines can't swim.

There’s 2 things going on here.

Output orientation - Is the output is similar to what a human would create if they were to think.

Process orientation - Is the machine actually thinking, when we say its thinking.

I met someone who once drew a circuit diagram from memory. However, they didn’t draw it from inputs, operations, to outputs. They started drawing from the upper left corner, and continued drawing to the lower right, adding lines, triangles and rectangles as need be.

Rote learning can help you pass exams. At some point, it’s a meaningless difference between the utility of “knowing” how engineering works, and being able to apply methods and provide a result.

This is very much the confusion at play here, so both points are true.

1) These tools do not “Think”, in any way that counts as human thinking

2) the output is often the same as what a human thinking, would create.

IF you are concerned with only the product, then what’s the difference? If you care about the process, then this isn’t thought.

To put it in a different context. If you are a consumer, do you care if the output was hand crafted by an artisan, or do you just need something that works.

If you are a producer in competition with others, you care if your competition is selling Knock offs at a lower price.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#204

Post by bilsbie »

Interestingly I just hit an example of this. Highly specific but I was asking about pickleball strategy and grok and Claude both couldn’t seem to understand you can’t aim at the opponent’s feet when you’re hitting up.

Just kept regurgitating internet advice and I couldn’t get it to understand the reasoning on why it was wrong.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#205

Post by bilsbie »

I wonders if there’s past symbolic reasoning research we could integrate into LLMs. They’re really good at parsing text and understanding the relationships between objects ie getting the “symbols” correct.

Maybe we plug into something like prolog (or other such strategies?)

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#206

Post by jqpabc123 »

bilsbie wrote: post #204

Interestingly I just hit an example of this. Highly specific but I was asking about pickleball strategy and grok and Claude both couldn’t seem to understand you can’t aim at the opponent’s feet when you’re hitting up. Just kept regurgitating internet advice and I couldn’t get it to understand the reasoning on why it was wrong.

Hey --- if the internet says it, it can't be wrong.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#207

Post by bilsbie »

jqpabc123 wrote: post #206
bilsbie wrote: post #204

Interestingly I just hit an example of this. Highly specific but I was asking about pickleball strategy and grok and Claude both couldn’t seem to understand you can’t aim at the opponent’s feet when you’re hitting up. Just kept regurgitating internet advice and I couldn’t get it to understand the reasoning on why it was wrong.

Hey --- if the internet says it, it can't be wrong.

In this case it found generic advice and was confusing itself.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#208

Post by jmogly »

piskov wrote: post #202

All "reasoning" models hit a complexity wall where they completely collapse to 0% accuracy. No matter how much computing power you give them, they can't solve harder problems. This research suggests we're not as close to AGI as the hype suggests. Current "reasoning" breakthroughs may be hitting fundamental walls that can't be solved by just adding more data or compute. Apple's researchers used controllable puzzle env…

I think this might be part of the reason Apple is “behind” on generative AI … LLMs have not really proven to be useful outside of relatively niche areas such as coding assistants, legal boiler plate and research, and maybe some data science/analysis which I’m less familiar with

Other “end user” facing use cases have so far been comically bad or possibly harmful, and they just don’t meet the quality bar for inclusion in Apple products, which as much as some people like to doo doo on them and say they have gotten worse, still have a very high expectations of quality and UX from customers.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#209

Post by MattRix »

bayindirh wrote: post #93
nmca wrote: post #77

Earlier quoted context omitted.

I saw your comment and counted — in May I took a Waymo thirty times.

Waymo is a popular argument in self-driving cars, and they do well. However, Waymo is Deep Blue of self-driving cars. Doing very well in a closed space . As a result of this geofencing, they have effectively exhausted their search space, hence they work well as a consequence of lack of surprises. AI works well when search space is limited, but General AI in any category needs to handle a vastly larger search space, a…

This view of Waymo doesn’t account for the fact that self driving is about a lot more than just taking the right roads. It has to deal with other drivers, construction, road closures, pedestrians, bikes, etc.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#210

Post by jqpabc123 »

bilsbie wrote: post #207
jqpabc123 wrote: post #206

Earlier quoted context omitted.

Hey --- if the internet says it, it can't be wrong.

In this case it found generic advice and was confusing itself.

That's one explanation.

Another could be that it simply has no real *understanding* of anything. It simply did a statistical comparison of the question to the available advice and picked the best match --- kinda what a search engine might do.

Expecting *understanding* from a synthetic, statistical process will often end in disappointment.

Post reply on HN