Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

ml-site.cdn-apple.com

Post reply on HN
Print view View original

251–260 of 276 posts

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#251

Post by fc417fc802 »

Swizec wrote: post #250
fc417fc802 wrote: post #191

Earlier quoted context omitted.

Ah yes the mythical strawman definition of AI that you can never seem to pin down, was never rigorous, and never enjoyed wide expert acceptance. It's on par with "well many people used to say, or at least so I've been told, that ...".

That’s the point: AI is a marketing term and always has been. The underlying tech changes with every hype wave. One of the first humanoid robots was an 18th century clockwork mechanism inside a porcelain doll that autonomously wrote out “Cogito Ergo Sum” in cursive with a pen. It was considered thought provoking at the time because it implied that some day machines could think. BBC video posted to reddit 10 years ago…

It certainly sees use as an ever shifting marketing term. That does not exclude it from being a useful technical term. Indeed if the misuse of a term by marketers was sufficient to rob a word of meaning then I doubt we'd have any means of communication left.

> It was considered thought provoking at the time because it implied that some day machines could think.

What constitutes "thinking"? That's approximately the same question as what qualifies as AGI. LLMs and RL seem to be the first time humanity has achieved anything that begins to resemble that but clearly both of those come up short ... at least so far.

Meanwhile I'm quite certain that a glorified PID loop (ie autopilot) does not qualify as machine learning (AI if you'd prefer). If someone wants to claim that it does then he's going to need to explain how his definition excludes mechanical clockwork.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#252

Post by make3 »

curious_cat_163 wrote: post #35

> Rather than standard benchmarks (e.g., math problems), we adopt controllable puzzle environments that let us vary complexity systematically Very clever, I must say. Kudos to folks who made this particular choice. > we identify three performance regimes: (1) low complexity tasks where standard models surprisingly outperform LRMs, (2) medium-complexity tasks where additional thinking in LRMs demonstrates advantage, a…

Using puzzles is not special or anything, it has been done a million times since before (and including) the LSTM paper (1997) https://www.bioinf.jku.at/publications/older/2604.pdf

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#253

Post by BriggyDwiggs42 »

imiric wrote: post #240
BriggyDwiggs42 wrote: post #135

Earlier quoted context omitted.

This approach to defining “true” intelligence seems flawed to me because of examples in biology where semantic understanding is in no way relevant to function. A slime mold solving a maze doesn’t even have a brain, yet it solves a problem to get food. There’s no knowing that it does that, no complex signal processing, no self-perception of purpose, but nevertheless it gets the food it needs. My response to that isn’t…

While it's debatable whether slime molds showcase intelligence, there's a substantial difference between its behavior and modern AI systems. The organism was never trained to traverse a maze. It simply behaves in the same way as it would in its natural habitat, seeking out food in this case, which we interpret as "solving" a human-made problem. In order to get an AI system to do the same we would have to "train" it o…

My alternative definition would be something like this. Intelligence is the capacity to solve problems, where a problem is defined contextually. This means that what is and is not intelligence is negotiable in situations where the problem itself is negotiable. If you have water solve a maze, then yes the water could be said to have intelligence, though that would be a silly way to put it. It’s more that intelligence is a material phenomenon, and things which seem like they should be incredibly stupid can demonstrate surprisingly intelligent behavior.

LLMs are leagues ahead of viruses or proteins or water. If you put an LLM into a code editor with access to error messages, it can solve a problem you create for it, much like water flowing through a maze. Does it learn or change? No, everything is already there in the structure of the LLM. Does it have agency? No, it’s a transparently deterministic mapping from input to output. Can it demonstrate intelligent behavior? Yes.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#254

Post by withinboredom »

fc417fc802 wrote: post #251
Swizec wrote: post #250

Earlier quoted context omitted.

That’s the point: AI is a marketing term and always has been. The underlying tech changes with every hype wave. One of the first humanoid robots was an 18th century clockwork mechanism inside a porcelain doll that autonomously wrote out “Cogito Ergo Sum” in cursive with a pen. It was considered thought provoking at the time because it implied that some day machines could think. BBC video posted to reddit 10 years ago…

It certainly sees use as an ever shifting marketing term. That does not exclude it from being a useful technical term. Indeed if the misuse of a term by marketers was sufficient to rob a word of meaning then I doubt we'd have any means of communication left. > It was considered thought provoking at the time because it implied that some day machines could think. What constitutes "thinking"? That's approximately the sa…

What do you think an executing LLM is? It’s basically a glorified PID loop. It isn’t learning anything new. It isn’t thinking about your conversation while you go take a poo.

And I think the point is that the definition doesn’t exclude pure mechanical devices since that’s exactly what a computer is.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#255

Post by imiric »

BriggyDwiggs42 wrote: post #253
imiric wrote: post #240

Earlier quoted context omitted.

While it's debatable whether slime molds showcase intelligence, there's a substantial difference between its behavior and modern AI systems. The organism was never trained to traverse a maze. It simply behaves in the same way as it would in its natural habitat, seeking out food in this case, which we interpret as "solving" a human-made problem. In order to get an AI system to do the same we would have to "train" it o…

My alternative definition would be something like this. Intelligence is the capacity to solve problems, where a problem is defined contextually. This means that what is and is not intelligence is negotiable in situations where the problem itself is negotiable. If you have water solve a maze, then yes the water could be said to have intelligence, though that would be a silly way to put it. It’s more that intelligence…

That's an interesting way of looking at it, though I do disagree. Mainly because, as you mention, it would be silly to claim that water is intelligent if it can be used to solve a problem. That would imply that any human-made tool is intelligent, which is borderline absurd.

This is why I think it's important that if we're going to call these tools intelligent, then they must follow the processes that humans do to showcase that intelligence. Scoring high on a benchmark is not a good indicator of this, in the same way that a human scoring high on a test isn't. It's just one convenient way we have of judging this, and a very flawed one at that.

Anyway, cheers for the discussion!

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#257

Post by intended »

imiric wrote: post #236
intended wrote: post #203

Earlier quoted context omitted.

There’s 2 things going on here. Output orientation - Is the output is similar to what a human would create if they were to think. Process orientation - Is the machine actually thinking, when we say its thinking. I met someone who once drew a circuit diagram from memory. However, they didn’t draw it from inputs, operations, to outputs. They started drawing from the upper left corner, and continued drawing to the lower…

> IF you are concerned with only the product, then what’s the difference? The difference is substantial. If the machine was actually thinking and it understood the meaning of its training data, it would be able to generate correct output based on logic, deduction, and association. We wouldn't need to feed it endless permutations of tokens so that it doesn't trip up when the input data changes slightly. This is the di…

That sentence, is from the perspective of someone only caring about the output.

The people who care about the process, have a different take, which I have also explained.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#258

Post by danaris »

hackinthebochs wrote: post #218
TheOtherHobbes wrote: post #212

Earlier quoted context omitted.

You don't need phenomenal consciousness. You need consistency. LLMs are not consistent. This is unarguable. They will produce a string of text that says they have solved a problem and/or done a thing when neither is true. And sometimes they will do it over and over, even when corrected. Your last paragraph admits this. Tokenisation on its own simply cannot represent reality accurately and reliably. It can be tweaked…

Consistency is a strange criteria seeing as humans aren't very consistent either. Intelligent beings can make mistakes.

This is a common category of error people commit when talking about LLMs.

"True, LLMs can't do X, but a lot of people don't do X well either!"

The problem is, when you say humans have trouble with X, what you mean is that human brains are fully capable of X, but sometimes they do, indeed, make mistakes. Or that some humans haven't trained their faculties for X very well, or whatever.

But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes.

These things are not comparable.

So, to your specific point: When an LLM is inconsistent, it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. When a human is inconsistent, it is because they got distracted, didn't learn enough about this particular subject, or otherwise made a mistake that they can, if their attention is drawn to it, recognize and correct.

LLMs cannot. They can only be told they made a mistake, which prompts them to try again (because that's the pattern that has been trained into them for what happens when told they made a mistake). But their next try won't have any better odds of being correct than their previous one.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#259

Post by danaris »

Swizec wrote: post #250
fc417fc802 wrote: post #191

Earlier quoted context omitted.

Ah yes the mythical strawman definition of AI that you can never seem to pin down, was never rigorous, and never enjoyed wide expert acceptance. It's on par with "well many people used to say, or at least so I've been told, that ...".

That’s the point: AI is a marketing term and always has been. The underlying tech changes with every hype wave. One of the first humanoid robots was an 18th century clockwork mechanism inside a porcelain doll that autonomously wrote out “Cogito Ergo Sum” in cursive with a pen. It was considered thought provoking at the time because it implied that some day machines could think. BBC video posted to reddit 10 years ago…

AI is multiple things.

AI is a marketing term for various kinds of machine learning applications.

AI is an academic field within computer science.

AI is the computer-controlled enemies you face in (especially, but not solely, offline) games.

This has been the case for decades now—especially the latter two.

Trying to claim that AI either "has always been" one particular thing, or "has now become" one particular thing, is always going to run into trouble because of this multiplicity. The one thing that AI "has always been" is multiple things.

Re: The Illusion of Thinking: Strengths and limitations of reasoning models [pdf]

#260

Post by bird0861 »

bwfan123 wrote: post #146
Too wrote: post #72

Earlier quoted context omitted.

The million dollar question is how far can one get on this trick. Maybe this is exactly how our own brains operate? If not, what fundamental building blocks are missing to get there.

> If not, what fundamental building blocks are missing to get there If I were to guess, the missing building block is the ability to abstract - which is the ability to create a symbol to represent something. Concrete example of abstraction is seen in the axioms of lambda calculus. 1) ability to posit a variable, 2) ability to define a function using said variable, and 3) the ability to apply functions to things. Abst…

No. It's not microtubules. Enough with the g-darn microtubules already. https://www.biorxiv.org/content/10.1101/712794v1

"We used an antimicrotubular agent (parbendazole) and disrupted microtubular dynamics in paramecium to see if microtubules are an integral part of information storage and processing in paramecium’s learning process. We observed that a partial allosteric modulator of GABA (midazolam) could disrupt the learning process in paramecium, but the antimicrotubular agent could not. Therefore, our results suggest that microtubules are probably not vital for the learning behavior in P. caudatum. Consequently, our results call for a further revisitation of the microtubular information processing hypothesis."

Post reply on HN