Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

141–150 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#141

Post by bubblyworld »

shannifin wrote: post #96
safety1st wrote: post #87

Earlier quoted context omitted.

I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work? I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministica…

Problem is, even with symbolic logic, reasoning is not completely deterministic. Whether one can get to a set of given axioms from a given proposition is sometimes undecidable.

I don't think this is really a problem. The general problem of finding a proof from some axioms to some formula is undecidable (in e.g. first order logic). But that doesn't tell you anything about specific cases, in the same way that we can easily tell whether some specific program halts, like this one:

"return 1"

Re: Is chain-of-thought AI reasoning a mirage?

#142

Post by blackbear_ »

lordnacho wrote: post #18
mentalgear wrote: post #5

> Whether AI reasoning is “real” reasoning or just a mirage can be an interesting question, but it is primarily a philosophical question. It depends on having a clear definition of what “real” reasoning is, exactly. It's pretty easy: causal reasoning. Causal, not statistic correlation only as LLM do, with or without "CoT".

What's stopping us from building an LLM that can build causal trees, rejecting some trees and accepting others based on whatever evidence it is fed? Or even a causal tool for an LLM agent that operates like what it does when you ask it about math and forwards the request to Wolfram.

In principle this is possible, modulo scalability concerns: https://arxiv.org/pdf/2506.06039

Perhaps this will one day become a new post-training task

Re: Is chain-of-thought AI reasoning a mirage?

#143

Post by wonnage »

safety1st wrote: post #87
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work? I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministica…

My impression of LLM “reasoning” is that works more like guardrails. Perhaps the space of possible responses to the initial prompt is huge and doesn’t exactly match any learned information. All the text generated during reasoning is high strength. So placing it in the context should hopefully guide answer generation towards something reasonable.

It’s the same idea as manually listing a bunch of possibly-useful facts in the prompt, but the LLM is able to generate plausible sounding text itself.

I feel like this relates to why LLM answers tend to be verbose too, it needs to put the words out there in order to stay coherent.

Re: Is chain-of-thought AI reasoning a mirage?

#144

Post by hannasm » 

  > these papers keep stapling on broad philosophical claims about whether models can “really reason” that are just completely unsupported by the content of the research.
From the scientific papers I've read almost every single research paper does this. What's the point of publishing a paper if it doesn't at least try to convince the readers that something award worthy has been learned?

Usually there may be some interesting ideas hidden in the data but the paper's methods and scope weren't even worthy of a conclusion to begin with. It's just one data point in the vast sea of scientific experimentation.

The conclusion feels to me like a cultural phenomenon and it's just a matter of survival for most authors. I have to imagine it was easier in the past.

"Does the flame burn green? Why yes it does..."

These days it's more like

"With my two hours of compute on the million dollar mainframe, my toy llm didn't seem to get there, YMMV"

Re: Is chain-of-thought AI reasoning a mirage?

#145

Post by Terr_ »

A regular "chatting" LLM is a document generator incrementally extending a story about a conversation between a human and a robot... And through that lens, I've been thinking "chain of thought" seems like basically the same thing but with a film noir styling-twist.

The LLM is trained to include an additional layer of "unspoken" text in the document, a source of continuity which substitutes for how the LLM has no other memories or goals to draw from.

"The capital of Assyria? Those were dangerous questions, especially in this kind of town. But rent was due, and the bottle in my drawer was empty. I took the case."

Re: Is chain-of-thought AI reasoning a mirage?

#146

Post by archaeans »

danans wrote: post #132
mrspuratic wrote: post #100

Earlier quoted context omitted.

On that topic, it seems backwards to me: intelligence is not emergent behaviour of language, rather the opposite.

Perception and interpretation can very much be influenced by language (Sapir-Wharf hypothesis), so to the extent that perception and interpretation influence intelligence, it's not clear that the relationship is only in one direction.

"It would be naïve to imagine that any analysis of experience is dependent on pattern expressed in language."

- Sapir

It's hard to take these discussions on cognition and intelligence seriously when there is so much lossy compression going on.

Re: Is chain-of-thought AI reasoning a mirage?

#147

Post by archaeans »

mirekrusin wrote: post #139
okasaki wrote: post #28

Earlier quoted context omitted.

Human babies are the same algorithm as adults.

This analogy would mean very large model that didn't finish training yet. Tiny model like this is more like doing study on fruit flies and extrapolating results to humans.

Every argument about LLMs of that is a variant of "humans same" is self defeating because it assumes a level of understanding of human cognition and the human brain that doesn't really exist outside of the imagination of people with a poor understanding of neuroscience.

Re: Is chain-of-thought AI reasoning a mirage?

#148

Post by archaeans »

modeless wrote: post #34

"The question [whether computers can think] is just as relevant and just as meaningful as the question whether submarines can swim." -- Edsger W. Dijkstra, 24 November 1983

And the often missed caveat is that we should only care about whether the software does what it is supposed to do.

Under that light, LLMs are just buggy and have been for years. Where is the LLM that does what it says it should do? "Hallucination" and "do they reason" are distractions. They fail. They're buggy.

Re: Is chain-of-thought AI reasoning a mirage?

#149

Post by archaeans »

hungmung wrote: post #63

Chain of thought is just a way of trying to squeeze more juice out of the lemon of LLM's; I suspect we're at the stage of running up against diminishing returns and we'll have to move to different foundational models to see any serious improvement.

The so-called "scaling laws" are expressing diminishing returns.

How is it that "if we grow resources used exponentially errors decrease linearly" ever seen as a good sign?

Re: Is chain-of-thought AI reasoning a mirage?

#150

Post by Terr_ »

potsandpans wrote: post #106
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

> It is not a "philosophical" (by which the author probably meant "practically inconsequential") question. I didn't take it that way. I suppose it depends on whether or not you believe philosophy is legitimate

> I suppose it depends on whether or not you believe philosophy is legitimate

The only way to declare philosophy illegitimate is to be using legitimate philosophy, so... :p

Post reply on HN