Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

21–30 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#21

Post by varelse »

vonneumannstan wrote: post #17
robviren wrote: post #4

I feel it is interesting but not what would be ideal. I really think if the models could be less linear and process over time in latent space you'd get something much more akin to thought. I've messed around with attaching reservoirs at each layer using hooks with interesting results (mainly over fitting), but it feels like such a limitation to have all model context/memory stuck as tokens when latent space is where…

>I feel it is interesting but not what would be ideal. I really think if the models could be less linear and process over time in latent space you'd get something much more akin to thought. Please stop, this is how you get AI takeovers.

[dead]

Re: Is chain-of-thought AI reasoning a mirage?

#22

Post by meindnoch »

gdbsjjdn wrote: post #11
empath75 wrote: post #6

One thing that LLMs have exposed is how much of a house of cards all of our definitions of "human mind"-adjacent concepts are. We have a single example in all of reality of a being that thinks like we do, and so all of our definitions of thinking are inextricably tied with "how humans think", and now we have an entity that does things which seem to be very like how we think, but not _exactly like it_, and a lot of ou…

Congratulations, you've invented philosophy.

We need to reinvent philosophy. With JSON this time.

Re: Is chain-of-thought AI reasoning a mirage?

#23

Post by adastra22 »

stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

Re: Is chain-of-thought AI reasoning a mirage?

#24

Post by stonemetal12 »

NitpickLawyer wrote: post #2

Finally! A good take on that paper. I saw that arstechnica article posted everywhere, and most of the comments are full of confirmation bias, and almost all of them miss the fineprint - it was tested on a 4 layer deep toy model. It's nice to read a post that actually digs deeper and offers perspectives on what might be a good finding vs. just warranting more research.

> it was tested on a 4 layer deep toy model

How do you see that impacting the results? It is the same algorithm just on a smaller scale. I would assume a 4 layer model would not be very good, but does reasoning improve it? Is there a reason scale would impact the use of reasoning?

Re: Is chain-of-thought AI reasoning a mirage?

#25

Post by adastra22 »

vonneumannstan wrote: post #17
robviren wrote: post #4

I feel it is interesting but not what would be ideal. I really think if the models could be less linear and process over time in latent space you'd get something much more akin to thought. I've messed around with attaching reservoirs at each layer using hooks with interesting results (mainly over fitting), but it feels like such a limitation to have all model context/memory stuck as tokens when latent space is where…

>I feel it is interesting but not what would be ideal. I really think if the models could be less linear and process over time in latent space you'd get something much more akin to thought. Please stop, this is how you get AI takeovers.

Citation seriously needed.

Re: Is chain-of-thought AI reasoning a mirage?

#26

Post by mdp2021 »

empath75 wrote: post #6

One thing that LLMs have exposed is how much of a house of cards all of our definitions of "human mind"-adjacent concepts are. We have a single example in all of reality of a being that thinks like we do, and so all of our definitions of thinking are inextricably tied with "how humans think", and now we have an entity that does things which seem to be very like how we think, but not _exactly like it_, and a lot of ou…

> which seem to be very like how we think

I would like to reassure you that we - we here - see LLMs are very much unlike us.

Re: Is chain-of-thought AI reasoning a mirage?

#27

Post by adastra22 »

empath75 wrote: post #14
gdbsjjdn wrote: post #11

Earlier quoted context omitted.

Congratulations, you've invented philosophy.

This is an obnoxious response. Of course I recognize that philosophy is the solution to this. What I am pointing out is that philosophy has not as of yet resolved these relatively new problems. The idea that non-human intelligences might exist is of course an old one, but that is different from having an actual (potentially) existing one to reckon with.

These are not new problems though.

Re: Is chain-of-thought AI reasoning a mirage?

#28

Post by okasaki »

stonemetal12 wrote: post #24
NitpickLawyer wrote: post #2

Finally! A good take on that paper. I saw that arstechnica article posted everywhere, and most of the comments are full of confirmation bias, and almost all of them miss the fineprint - it was tested on a 4 layer deep toy model. It's nice to read a post that actually digs deeper and offers perspectives on what might be a good finding vs. just warranting more research.

> it was tested on a 4 layer deep toy model How do you see that impacting the results? It is the same algorithm just on a smaller scale. I would assume a 4 layer model would not be very good, but does reasoning improve it? Is there a reason scale would impact the use of reasoning?

Human babies are the same algorithm as adults.

Re: Is chain-of-thought AI reasoning a mirage?

#30

Post by stonemetal12 »

glial wrote: post #9
mentalgear wrote: post #5

> Whether AI reasoning is “real” reasoning or just a mirage can be an interesting question, but it is primarily a philosophical question. It depends on having a clear definition of what “real” reasoning is, exactly. It's pretty easy: causal reasoning. Causal, not statistic correlation only as LLM do, with or without "CoT".

Correct me if I'm wrong, I'm not sure it's so simple. LLMs are called causal models in the sense that earlier tokens "cause" later tokens, that is, later tokens are causally dependent on what the earlier tokens are. If you mean deterministic rather than probabilistic, even Pearl-style causal models are probabilistic. I think the author is circling around the idea that their idea of reasoning is to produce statements…

Smoking increases the risk of getting cancer significantly. We say Smoking causes Cancer. Causal reasoning can be probabilistic.

LLMs are not causal reasoning because there are no facts, only tokens. For the most part you can't ask LLMs how they came to an answer, because it doesn't know.

Post reply on HN