Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

71–80 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#71

Post by wizzwizz4 »

griffzhowl wrote: post #60
modeless wrote: post #34

"The question [whether computers can think] is just as relevant and just as meaningful as the question whether submarines can swim." -- Edsger W. Dijkstra, 24 November 1983

I don't agree with the parallel. Submarines can move through water - whether you call that swimming or not isn't an interesting question, and doesn't illuminate the function of a submarine. With thinking or reasoning, there's not really a precise definition of what it is, but we nevertheless know that currently LLMs and machines more generally can't reproduce many of the human behaviours that we refer to as thinking.…

https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898... provides the context. I haven't re-read it in the last month, but I'm pretty sure you've correctly identified Dijkstra's point.

Re: Is chain-of-thought AI reasoning a mirage?

#72

Post by blargey »

energy123 wrote: post #52
stonemetal12 wrote: post #39

Earlier quoted context omitted.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?

s/random/statistically-likely/g

Reducing the distance of each statistical leap improves “performance” since you would avoid failure modes that are specific to the largest statistical leaps, but it doesn’t change the underlying mechanism. Reasoning models still “hallucinate” spectacularly even with “shorter” gaps.

Re: Is chain-of-thought AI reasoning a mirage?

#73

Post by ikari_pl »

blargey wrote: post #72
energy123 wrote: post #52

Earlier quoted context omitted.

Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?

s/random/statistically-likely/g Reducing the distance of each statistical leap improves “performance” since you would avoid failure modes that are specific to the largest statistical leaps, but it doesn’t change the underlying mechanism. Reasoning models still “hallucinate” spectacularly even with “shorter” gaps.

What's wrong with statistically likely?

If I ask you what's 2+2, there's a single answer I consider much more likely than others.

Sometimes, words are likely because they are grounded in ideas and facts they represent.

Re: Is chain-of-thought AI reasoning a mirage?

#74

Post by cyanydeez »

stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

They should call them Fuzzing models. They're just running through varioous iterations of the context until they hit a token that trips them out.

Re: Is chain-of-thought AI reasoning a mirage?

#75

Post by LudwigNagasena »

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language.

That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "think" and "reason" without outputting a single "word".

> Whether AI reasoning is “real” reasoning or just a mirage can be an interesting question, but it is primarily a philosophical question. It depends on having a clear definition of what “real” reasoning is, exactly.

It is not a "philosophical" (by which the author probably meant "practically inconsequential") question. If the whole reasoning business is just rationalization of pre-computed answers or simply a means to do some computations because every token provides only a fixed amount of computation to update the model's state, then it doesn't make much sense to focus on improving the quality of chain-of-thought output from human POV.

Re: Is chain-of-thought AI reasoning a mirage?

#76

Post by benreesman »

stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

People will go to extremely great lengths to debate the appropriate analogy for how these things work, which is fun I guess but in a "get high with a buddy" sense at least to my taste.

Some of how they work is well understood (a lot now, actually), some of the outcomes are still surprising.

But we debate both the well understood parts and the surprising parts both with the wrong terminology borrowed from pretty dubious corners of pop cognitive science, and not with terminology appropriate to the new and different thing! It's nothing like a brain, it's a new different thing. Does it think or reason? Who knows pass the blunt.

They do X performance on Y task according to Z eval, that's how you discuss ML model capability if you're persuing understanding rather than fundraising or clicks.

Re: Is chain-of-thought AI reasoning a mirage?

#77

Post by kelipso »

pixl97 wrote: post #61
kelipso wrote: post #53

Earlier quoted context omitted.

No, people make logical connections, make inferences, make sure all of it fits together without logical errors, etc.

These people you're talking about must be rare online, as human communication is pretty rife with logical errors.

You shouldn’t be basing your entire worldview around the lowest common denominator. All kinds of writers like blog writers, novelists, scriptwriters, technical writers, academics, poets, lawyers, philosophers, mathematicians, and even teenage fan fiction writers do what I said above routinely.

Re: Is chain-of-thought AI reasoning a mirage?

#78

Post by skywhopper »

I mostly agree with the point the author makes that "it doesn't matter". But then again, it does matter, because LLM-based products are marketed based on "IT CAN REASON!" And so, while it may not matter, per se, how an LLM comes up with its results, to the extent that people choose to rely on LLMs because of marketing pitches, it's worth pushing back on those claims if they are overblown, using the same frame that the marketers use.

That said, this author says this question of whether models "can reason" is the least interesting thing to ask. But I think the least interesting thing you can do is to go around taking every complaint about LLM performance and saying "but humans do the exact same thing!" Which is often not true, but again, doesn't matter.

Re: Is chain-of-thought AI reasoning a mirage?

#79

Post by ActionHank »

brunokim wrote: post #69

I'm unconvinced by the article criticism's, given they also employ their feels and few citations. > I appreciate that research has to be done on small models, but we know that reasoning is an emergent capability! (...) Even if you grant that what they’re measuring is reasoning, I am profoundly unconvinced that their results will generalize to a 1B, 10B or 100B model. A fundamental part of applied research is simplify…

I think that when you are arguing logic and reason with a group who became really attached to the term vibe-coding you've likely already lost.

Re: Is chain-of-thought AI reasoning a mirage?

#80

Post by kazinator »

LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

Not all reasoning requires language. Symbolic reasoning uses language.

Real-time spatial reasoning like driving a car and not hitting things does not seem linguistic.

Figuring out how to rotate a cabinet so that it will clear through a stairwell also doesn't seem like it requires language, only to communicate the solution to someone else (where language can turn into a hindrance, compared to a diagram or model).

Post reply on HN