The Illusion of Thinking: Strengths and limitations of reasoning models [pdf] • Hacker News Board

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#161

Post by naasking » Sat, Jun 07, 2025, 6:45 PM UTC

jackdoe wrote: post #141

dmos62 wrote: post #139

Earlier quoted context omitted.

What's your approach?

For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers https://punkx.org/jackdoe/30.html and so on. And of course Yannic Kilcher[4], and also listening in on th…

I think you'll need something like Meta's Large Concept Models to get past the language and token barrier.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#162

Post by naasking » Sat, Jun 07, 2025, 6:51 PM UTC

Kon5ole wrote: post #83

imiric wrote: post #71

Earlier quoted context omitted.

> I think the intuition the authors are trying to capture is that they believe the models are omniscient, but also dim-witted. We keep assigning adjectives to this technology that anthropomorphize the neat tricks we've invented. There's nothing "omniscient" or "dim-witted" about these tools. They have no wit. They do not think or reason. All Large "Reasoning" Models do is generate data that they use as context to gen…

>They have no wit. They do not think or reason. Computers can't think and submarines can't swim.

"Can't" is a pretty strong word, effectively entailing "never". Never is a long time to believe computers can't think.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#163

Post by naasking » Sat, Jun 07, 2025, 6:54 PM UTC

actinium226 wrote: post #24

Man, remember when everyone was like 'AGI just around the corner!' Funny how well the Gartner hype cycle captures these sorts of things

Interpreting "just around the corner" as "this year" sounds like your error. Most projections are are years out, at least.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#164

Post by PantaloonFlames » Sat, Jun 07, 2025, 6:57 PM UTC

latchup wrote: post #40

actinium226 wrote: post #24

Man, remember when everyone was like 'AGI just around the corner!' Funny how well the Gartner hype cycle captures these sorts of things

To be fair, the technology sigmoid curve rises fastest just before its inflection point, so it is hard to predict at what point innovation slows down due to its very nature. The first Boeing 747 was rolled out in 1968, only 65 years after the first successful heavier-than-air flight. If you told people back then that not much will fundamentally change in civil aviation over the next 57 years, no one would have believ…

And not just in aviation. Consider what aviation did to make the world smaller. Huge 2nd order changes. The COVID-19 pandemic would not have happened the way it did, if there were no Boeing or Airbus.

Big hard-to-predict changes ahead.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#165

Post by antithesizer » Sat, Jun 07, 2025, 6:58 PM UTC

stephc_int13 wrote: post #132

Human language is far from perfect as a cognitive tool but still serves us well because it is not foundational. We use it both for communication and some reasoning/planning as a high level layer. I strongly believe that human language is too weak (vague, inconsistent, not expressive enough etc.) to replace interactions with the world as a basis to build strong cognition. We're easily fooled by the results of LLM/LRM…

Language mediates those interactions with the world. There is no unmediated interaction with the world. Those moments when one feels most directly in contact with reality, that is when one is so deep down inside language that one cannot see daylight at all.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#166

Post by PantaloonFlames » Sat, Jun 07, 2025, 7:02 PM UTC

Marazan wrote: post #116

mirekrusin wrote: post #91

Earlier quoted context omitted.

I remember "stochastic parrot" and people saying it's fancy markov chain/dead end. You don't hear them much after roughly agentic coding appeared.

Spicy autocomplete is still spicy autocomplete

Yours seems like a c.2023 perspective of coding assistants. These days it’s well beyond autocomplete and “generate a function that returns the numbers from the Fibonacci sequence.”

But I would think that would be well understood here.

How can you reduce what is currently possible to spicy autocomplete? That seems pretty dismissive, so much so that I wonder if it is motivated reasoning on your part.

I’m not saying it’s good or bad; I’m just saying the capability is well beyond auto complete.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#167

Post by bufferoverflow » Sat, Jun 07, 2025, 7:08 PM UTC

jackdoe wrote: post #111

I think one of the reason we are confused about what LLMs can do is because they use language. And we look at the "reasoning traces" and the tokens there look human, but what is actually happening is very alien to us, as shown by "Biology of Large Language Models"[1] and "Safety Alignment Should Be Made More Than Just a Few Tokens Deep"[2] I am struggling a lot to see what the tech can and can not do, particularly de…

> we are confused about what LLMs can do is because they use language.

But they can also do math, logic, music notation, write code, LaTeX, SVG, etc.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#168

Post by throwaway71271 » Sat, Jun 07, 2025, 7:10 PM UTC

naasking wrote: post #161

jackdoe wrote: post #141

Earlier quoted context omitted.

For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers https://punkx.org/jackdoe/30.html and so on. And of course Yannic Kilcher[4], and also listening in on th…

I think you'll need something like Meta's Large Concept Models to get past the language and token barrier.

I think you are right, even if I beleve next token prediction can work, I dont think it can happen in this autoregressive way where we fully collapse the token to feed it back in. Can you imagine how much is lost from each torch.multinomial?

Maybe the way forward is in LCM or go JEPA, therwise, as this Apple paper suggests, we will just keep pushing the "pattern matching" further, maybe we get some sort of phase transition at some point or maybe we have to switch architecture, we will see. It could be that things change when we get physical multimodality and real world experience, I dont know.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#169

Post by throwaway71271 » Sat, Jun 07, 2025, 7:51 PM UTC

bufferoverflow wrote: post #167

jackdoe wrote: post #111

I think one of the reason we are confused about what LLMs can do is because they use language. And we look at the "reasoning traces" and the tokens there look human, but what is actually happening is very alien to us, as shown by "Biology of Large Language Models"[1] and "Safety Alignment Should Be Made More Than Just a Few Tokens Deep"[2] I am struggling a lot to see what the tech can and can not do, particularly de…

> we are confused about what LLMs can do is because they use language. But they can also do math, logic, music notation, write code, LaTeX, SVG, etc.

as this paper shows, it sees they can do tower of hanoi as well, up to a certain point that is.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#170

Post by farts_mckensy » Sat, Jun 07, 2025, 8:18 PM UTC

epolanski wrote: post #14

farts_mckensy wrote: post #3

[flagged]

Your tone is a bit odd, but I agree with you on this 100%. Without properly defining what thinking is and what is not, you can't discuss it's properties, let alone discuss which entity manifests it or not.

My tone is odd? Never heard that before in my life. Definitely wasn't an problem for me growing up.