Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

131–140 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#131

Post by geraneum »

farts_mckensy wrote: post #5
Terr_ wrote: post #4

Earlier quoted context omitted.

Equivalent: "I'm so sick of these [atheist] cretins proclaiming that [god doesn't exist], and when you drill down to first principles, these same people do not have a rigorous definition of [god] in the first place!" That's nonsense, because the people obligated to furnish a "rigorous definition" are the people who make the positive claim that something specific is happening. Also, the extraordinary claims are the on…

Your god analogy is clumsy in this case. We aren't talking about something fantastical here. Reasoning is not difficult to define. We can go down that road if you'd like. Rather, the problem is, once you do define it, you will quickly find that LLMs are capable of it. And that makes human exceptionalists a bit uncomfortable.

> Rather, the problem is, once you do define it, you will quickly find that LLMs are capable of it.

That’s really not what’s happening though. People who claim LLMs can do X and Y often don’t even understand how LLMs work. The opposite is also true. They just open a prompt and get an output and shout Eureka. Of course not everyone is like this, but majority are. It’s similar to what we think about thinking itself. You read these comments and everyone is an expert on human brain and how it works. It’s fascinating.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#132

Post by stephc_int13 »

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer.

I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition.

We're easily fooled by the results of LLM/LRM models because we typically use language fluency and knowledge retrieval as a proxy benchmark for intelligence among our peers.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#133

Post by grugagag »

imiric wrote: post #92
tim333 wrote: post #84

Earlier quoted context omitted.

>There's nothing "omniscient" or "dim-witted" about these tools I disagree in that that seems quite a good way of describing them. All language is a bit inexact. Also I don't buy we are no closer to AI than ten years ago - there seem lots going on. Just because LLMs are limited doesn't mean we can't find or add other algorithms - I mean look at alphaevolve for example https://www.technologyreview.com/2025/05/14/11164…

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

I keep on trying this wolf cabbage goat problem with various permutations, let’s say just a wolf and a cabbage, no goat mentioned. At some step the got materializes in the answer. I tell it there is no goat and yet it answers again and the goat is there.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#134

Post by wavemode »

jbentley1 wrote: post #117

Is Apple failing at AI so they just put all their R&D towards convincing themselves it isn't important?

I get the sense that many of the AI features shoved into consumer products recently have been marketed more towards investors than users. The companies are basically advertising that they're "keeping up" with the competition, meanwhile the features themselves receive mixed-to-poor reviews and are never capable of all the things advertised. So it seems to me that all of Apple, Google, Meta, Microsoft, and Samsung are currently "failing" at AI in exactly the same ways. If Apple is trying to start going a different direction that seems like a good sign.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#135

Post by BriggyDwiggs42 »

imiric wrote: post #92
tim333 wrote: post #84

Earlier quoted context omitted.

>There's nothing "omniscient" or "dim-witted" about these tools I disagree in that that seems quite a good way of describing them. All language is a bit inexact. Also I don't buy we are no closer to AI than ten years ago - there seem lots going on. Just because LLMs are limited doesn't mean we can't find or add other algorithms - I mean look at alphaevolve for example https://www.technologyreview.com/2025/05/14/11164…

> I figure it's hard to argue that that is not at least somewhat intelligent? The fact that this technology can be very useful doesn't imply that it's intelligent. My argument is about the language used to describe it, not about its abilities. The breakthroughs we've had is because there is a lot of utility from finding patterns in data which humans aren't very good at. Many of our problems can be boiled down to this…

This approach to defining “true” intelligence seems flawed to me because of examples in biology where semantic understanding is in no way relevant to function. A slime mold solving a maze doesn’t even have a brain, yet it solves a problem to get food. There’s no knowing that it does that, no complex signal processing, no self-perception of purpose, but nevertheless it gets the food it needs. My response to that isn’t to say the slime mold has no intelligence, it’s to widen the definition of intelligence to include the mold. In other words, intelligence is something one does rather than has; it’s not the form but the function of the thing. Certainly LLMs lack anything in any way resembling human intelligence, they even lack brains, but they demonstrate a capacity to solve problems I don’t think is unreasonable to label intelligent behavior. You can put them in some mazes and LLMs will happen to solve them.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#137

Post by trc001 »

Jensson wrote: post #127
withinboredom wrote: post #121

Earlier quoted context omitted.

AI encompasses a wide range of algorithms and techniques; not just LLMs or neural nets. Also, it is worth pointing out that the definition of AI has changed drastically over the last few years and narrowed pretty significantly. If you’re viewing the definition from the 80–90’s, most of what we call "automation" today would have been considered AI.

Autopilots were a thing before computers were a thing, you can implement one using mechanics and control theory. So no, traditional autopilots are not AI under any reasonable definition, otherwise every single machine we build would be considered AI as almost all machines has some form of control systems in them, for example is your microwave clock an AI? So I'd argue any algorithm that comes from control theory is n…

Uh, the bellman equation was first used for control theory and is the foundation of modern reinforcement learning... so wouldn't that imply LLMs "come from" control theory?

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#138

Post by squidproquo »

stephc_int13 wrote: post #132

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer. I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition. We're easily fooled by the results of LLM/LRM…

Agree with this. Human language is also not very information-dense; there is a lot of redundancy and uninformative repetition of words.

I also wonder about the compounding effects of luck and survivorship bias when using these systems. If you model a series of interactions with these systems probabilistically, as a series of failure/success modes, then you are bound to get a sub-population of users (of LLM/LLRMs) that will undoubtedly have “fantastic” results. This sub-population will then espouse and promote the merits of the system. There is clearly something positive these models do, but how much of the “success” is just luck.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#139

Post by dmos62 »

throwaway71271 wrote: post #118
dmos62 wrote: post #114

Earlier quoted context omitted.

> how to build systems where the whole is bigger than the sum of its parts A bit tangential, but I look at programming as inherently being that. Every task I try to break down into some smaller tasks that together accomplish something more. That leads me to think that, if you structure the process of programming right, you will only end up solving small, minimally interwined problems. Might sound far-fetched, but I t…

> And, even the dumber LLMs would slot in naturally into such a process That is what I am struggling with, it is really easy at the moment to slot LLM and make everything worse. Mainly because its output is coming from torch.multinomial with all kinds of speculative decoding and quantizations and etc. But I am convinced it is possible, just not the way I am doing it right now, thats why I am spending most of my time…

What's your approach?

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#140

Post by overu589 »

jackdoe wrote: post #111

I think one of the reason we are confused about what LLMs can do is because they use language. And we look at the "reasoning traces" and the tokens there look human, but what is actually happening is very alien to us, as shown by "Biology of Large Language Models"[1] and "Safety Alignment Should Be Made More Than Just a Few Tokens Deep"[2] I am struggling a lot to see what the tech can and can not do, particularly de…

> build systems where the whole is bigger than the sum of its parts.

Any “product” can be thought of this way.

Of systems there are many systems nested within systems, yet a simple singular order “emerges”, usually it is the designed intended function.

The trick to discerning systems lies in their relationships.

Actors through interfaces have a relationship (usually more than one so think of each relationship as its own system dynamic.)

A relationship is where the magic happens, usually a process with work being done (therefore interface inputs must account for this balance.)

Vectors. Vectors I am thinking are the real intellectual and functional mechanisms. Most systems process inputs of potential (“energy”) control signal (“information”) and assets (other actors for nested systems). Processes do the work of adding vector solutions [for some other problem] for whatever the output is.

That’s the topology as I am seeing it.

Post reply on HN