Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

31–40 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#31

Post by mdp2021 »

mentalgear wrote: post #5

> Whether AI reasoning is “real” reasoning or just a mirage can be an interesting question, but it is primarily a philosophical question. It depends on having a clear definition of what “real” reasoning is, exactly. It's pretty easy: causal reasoning. Causal, not statistic correlation only as LLM do, with or without "CoT".

> causal reasoning

You have missed the foundation: before dynamics, being. Before causal reasoning you have deep definition of concepts. Causality is "below" that.

Re: Is chain-of-thought AI reasoning a mirage?

#32

Post by suddenlybananas »

lordnacho wrote: post #18
mentalgear wrote: post #5

> Whether AI reasoning is “real” reasoning or just a mirage can be an interesting question, but it is primarily a philosophical question. It depends on having a clear definition of what “real” reasoning is, exactly. It's pretty easy: causal reasoning. Causal, not statistic correlation only as LLM do, with or without "CoT".

What's stopping us from building an LLM that can build causal trees, rejecting some trees and accepting others based on whatever evidence it is fed? Or even a causal tool for an LLM agent that operates like what it does when you ask it about math and forwards the request to Wolfram.

>What's stopping us from building an LLM that can build causal trees, rejecting some trees and accepting others based on whatever evidence it is fed?

Exponential time complexity.

Re: Is chain-of-thought AI reasoning a mirage?

#33

Post by deadbabe »

empath75 wrote: post #14
gdbsjjdn wrote: post #11

Earlier quoted context omitted.

Congratulations, you've invented philosophy.

This is an obnoxious response. Of course I recognize that philosophy is the solution to this. What I am pointing out is that philosophy has not as of yet resolved these relatively new problems. The idea that non-human intelligences might exist is of course an old one, but that is different from having an actual (potentially) existing one to reckon with.

Non-human intelligences have always existed in the form of animals.

Animals do not have spoken language the way humans do, so their thoughts aren’t really composed of sentences. Yet, they have intelligence and can reason about their world.

How could we build an AGI that doesn’t use language to think at all? We have no fucking clue and won’t for a while because everyone is chasing the mirage created by LLMs. AI winter will come and we’ll sit around waiting for the next big innovation. Probably some universal GOAP with deeply recurrent neural nets.

Re: Is chain-of-thought AI reasoning a mirage?

#35

Post by bongodongobob »

stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

And yet it improves their problem solving ability.

Re: Is chain-of-thought AI reasoning a mirage?

#36

Post by skybrian »

Mathematical reasoning does sometimes require correct calculations, and if you get them wrong your answers will be wrong. I wouldn’t want someone doing my taxes to be bad at calculation or bad at finding mistakes in calculation.

It would be interesting to see if this study’s results can be reproduced in a more realistic setting.

Re: Is chain-of-thought AI reasoning a mirage?

#37

Post by azrazalea_debt »

stonemetal12 wrote: post #24
NitpickLawyer wrote: post #2

Finally! A good take on that paper. I saw that arstechnica article posted everywhere, and most of the comments are full of confirmation bias, and almost all of them miss the fineprint - it was tested on a 4 layer deep toy model. It's nice to read a post that actually digs deeper and offers perspectives on what might be a good finding vs. just warranting more research.

> it was tested on a 4 layer deep toy model How do you see that impacting the results? It is the same algorithm just on a smaller scale. I would assume a 4 layer model would not be very good, but does reasoning improve it? Is there a reason scale would impact the use of reasoning?

A lot of current LLM work is basically emergent behavior. They use a really simple core algorithm and scale it up, and interesting things happen. You can read some of anthropic's recent papers to see some of this, like: They didn't expect LLMs could "lookahead" when writing poetry. However, when they actually went in and watched what was happening (there's details on how this "watching" works on their blog/in their studies) they found the LLM actually was planning ahead! That's emergent behavior, they didn't design it to do that, it just started doing due to the complexity of the model.

If (BIG if) we ever do see actual AGI, it is likely to work like this. It's unlikely we're going to make AGI by designing some grand Cathedral of perfect software, it is more likely we are going to find the right simple principles to scale big enough to have AGI emerge. This is similar.

Re: Is chain-of-thought AI reasoning a mirage?

#38

Post by empath75 »

mdp2021 wrote: post #26
empath75 wrote: post #6

One thing that LLMs have exposed is how much of a house of cards all of our definitions of "human mind"-adjacent concepts are. We have a single example in all of reality of a being that thinks like we do, and so all of our definitions of thinking are inextricably tied with "how humans think", and now we have an entity that does things which seem to be very like how we think, but not _exactly like it_, and a lot of ou…

> which seem to be very like how we think I would like to reassure you that we - we here - see LLMs are very much unlike us.

Yes I very much understand that most people do not think that LLMs think or understand like we do, but it is _very difficult_ to prove that that is the case, using any test which does not also exclude a great deal of people. And that is because "thinking like we do" is not at all a well-defined concept.

Re: Is chain-of-thought AI reasoning a mirage?

#39

Post by stonemetal12 »

adastra22 wrote: post #23
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.
Post reply on HN