Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

81–90 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#81

Post by CooCooCaCha »

stonemetal12 wrote: post #39
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

Reasoning is also about processing facts.

Re: Is chain-of-thought AI reasoning a mirage?

#82

Post by vmg12 »

LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

Solutions to some of the hardest problems I've had have only come after a night of sleep or when I'm out on a walk and I'm not even thinking about the problem. Maybe what my brain was doing was something different from reasoning?

Re: Is chain-of-thought AI reasoning a mirage?

#83

Post by gilbetron »

empath75 wrote: post #6

One thing that LLMs have exposed is how much of a house of cards all of our definitions of "human mind"-adjacent concepts are. We have a single example in all of reality of a being that thinks like we do, and so all of our definitions of thinking are inextricably tied with "how humans think", and now we have an entity that does things which seem to be very like how we think, but not _exactly like it_, and a lot of ou…

I agree 100% with you. I'm most excited about LLMs because they seem to capture at least some aspect of intelligence, and that's amazing given how much long it took to get here. It's exciting that we don't just understand it.

I see people say, "LLMs aren't human intelligence", but instead, I really feel that it shows that many people, and much of what we do, probably is like an LLM. Most people just hallucinate their way through a conversation, they certainly don't reason. Reasoning is incredibly rare.

Re: Is chain-of-thought AI reasoning a mirage?

#84

Post by blargey »

ikari_pl wrote: post #73
blargey wrote: post #72

Earlier quoted context omitted.

s/random/statistically-likely/g Reducing the distance of each statistical leap improves “performance” since you would avoid failure modes that are specific to the largest statistical leaps, but it doesn’t change the underlying mechanism. Reasoning models still “hallucinate” spectacularly even with “shorter” gaps.

What's wrong with statistically likely? If I ask you what's 2+2, there's a single answer I consider much more likely than others. Sometimes, words are likely because they are grounded in ideas and facts they represent.

> Sometimes, words are likely because they are grounded in ideas and facts they represent.

Yes, and other times they are not. I think the failure modes of a statistical model of a communicative model of thought are unintuitive enough without any added layers of anthropomorphization, so there remains some value in pointing it out.

Re: Is chain-of-thought AI reasoning a mirage?

#86

Post by andybak »

vmg12 wrote: post #82
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

Solutions to some of the hardest problems I've had have only come after a night of sleep or when I'm out on a walk and I'm not even thinking about the problem. Maybe what my brain was doing was something different from reasoning?

This is a very important point and mostly absent from the conversation.

We have many words that almost mean the same thing or can mean ment different things - and conversations about intelligence and consciousness are riddled with them.

Re: Is chain-of-thought AI reasoning a mirage?

#87

Post by safety1st »

LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work?

I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministically. This is the Platonic ideal of reasoning that the author of the post pooh-poohs, I guess, but it seems to me to be the whole point of reasoning; reasoning is the application of logic in evaluating a proposition. The LLM might be trained to generate elements of the proposition, but it's too random to apply logic.

Re: Is chain-of-thought AI reasoning a mirage?

#88

Post by dmacfour »

Enginerrrd wrote: post #66
phailhaus wrote: post #51

Earlier quoted context omitted.

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. It's true that if it's not producing text, there is no thinking involved, but it is absolutely NOT clear that the attention block isn't holding state and modeling something as it works to produce text predictions. In fact, I can't think of a way t…

> The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute.

How so? Transformers are state space models.

Re: Is chain-of-thought AI reasoning a mirage?

#89

Post by ThrowawayTestr »

stonemetal12 wrote: post #39
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

Have you read the chain of thought output from reasoning models? That's not what it does.

Re: Is chain-of-thought AI reasoning a mirage?

#90

Post by modeless »

griffzhowl wrote: post #60
modeless wrote: post #34

"The question [whether computers can think] is just as relevant and just as meaningful as the question whether submarines can swim." -- Edsger W. Dijkstra, 24 November 1983

I don't agree with the parallel. Submarines can move through water - whether you call that swimming or not isn't an interesting question, and doesn't illuminate the function of a submarine. With thinking or reasoning, there's not really a precise definition of what it is, but we nevertheless know that currently LLMs and machines more generally can't reproduce many of the human behaviours that we refer to as thinking.…

It is strange that you started your comment with "I don't agree". The rest of the comment demonstrates that you do agree.
Post reply on HN