Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

51–60 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#51

Post by phailhaus »

adastra22 wrote: post #23
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist.

Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

Re: Is chain-of-thought AI reasoning a mirage?

#52

Post by energy123 »

stonemetal12 wrote: post #39
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?

Re: Is chain-of-thought AI reasoning a mirage?

#53

Post by kelipso »

adastra22 wrote: post #23
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

No, people make logical connections, make inferences, make sure all of it fits together without logical errors, etc.

Re: Is chain-of-thought AI reasoning a mirage?

#54

Post by double0jimb0 »

phailhaus wrote: post #51
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

So exactly what language/paradigm is this brain modeling the problem within?

Re: Is chain-of-thought AI reasoning a mirage?

#55

Post by phailhaus »

double0jimb0 wrote: post #54
phailhaus wrote: post #51

Earlier quoted context omitted.

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

So exactly what language/paradigm is this brain modeling the problem within?

We literally don't know. We don't understand how the brain stores concepts. It's not necessarily language: there are people that do not have an internal monologue, and yet they are still capable of higher level thinking.

Re: Is chain-of-thought AI reasoning a mirage?

#56

Post by kelipso »

energy123 wrote: post #52
stonemetal12 wrote: post #39

Earlier quoted context omitted.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?

Technically random can have probabilities associated with them.. Casual speech, random means equal probabilities, or we don’t know the probabilities. But for LLM token output, it does estimate the probabilities.

Re: Is chain-of-thought AI reasoning a mirage?

#57

Post by chrisweekly »

phailhaus wrote: post #55
double0jimb0 wrote: post #54

Earlier quoted context omitted.

So exactly what language/paradigm is this brain modeling the problem within?

We literally don't know. We don't understand how the brain stores concepts. It's not necessarily language: there are people that do not have an internal monologue, and yet they are still capable of higher level thinking.

Rilke: "There is a depth of thought untouched by words, and deeper still a depth of formless feeling untouched by thought."

Re: Is chain-of-thought AI reasoning a mirage?

#60

Post by griffzhowl »

modeless wrote: post #34

"The question [whether computers can think] is just as relevant and just as meaningful as the question whether submarines can swim." -- Edsger W. Dijkstra, 24 November 1983

I don't agree with the parallel. Submarines can move through water - whether you call that swimming or not isn't an interesting question, and doesn't illuminate the function of a submarine.

With thinking or reasoning, there's not really a precise definition of what it is, but we nevertheless know that currently LLMs and machines more generally can't reproduce many of the human behaviours that we refer to as thinking.

The question of what tasks machines can currently accomplish is certainly meaningful, if not urgent, and the reason LLMs are getting so much attention now is that they're accomplishing tasks that machines previously couldn't do.

To some extent there might always remain a question about whether we call what the machine is doing "thinking" - but that's the uninteresting verbal question. To get at the meaningful questions we might need a more precise or higher resolution map of what we mean by thinking, but the crucial element is what functions a machine can perform, what tasks it can accomplish, and whether we call that "thinking" or not doesn't seem important.

Maybe that was even Dijkstra's point, but it's hard to tell without context...

Post reply on HN