Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

61–70 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#61

Post by pixl97 »

kelipso wrote: post #53
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

No, people make logical connections, make inferences, make sure all of it fits together without logical errors, etc.

These people you're talking about must be rare online, as human communication is pretty rife with logical errors.

Re: Is chain-of-thought AI reasoning a mirage?

#62

Post by gdbsjjdn »

empath75 wrote: post #14
gdbsjjdn wrote: post #11

Earlier quoted context omitted.

Congratulations, you've invented philosophy.

This is an obnoxious response. Of course I recognize that philosophy is the solution to this. What I am pointing out is that philosophy has not as of yet resolved these relatively new problems. The idea that non-human intelligences might exist is of course an old one, but that is different from having an actual (potentially) existing one to reckon with.

> Writings on metacognition date back at least as far as two works by the Greek philosopher Aristotle (384–322 BC): On the Soul and the Parva Naturalia

We built a box that spits out natural language and tricks humans into believing it's conscious. The box itself actually isn't that interesting, but the human side of the equation is.

Re: Is chain-of-thought AI reasoning a mirage?

#64

Post by mdp2021 »

pixl97 wrote: post #61
kelipso wrote: post #53

Earlier quoted context omitted.

No, people make logical connections, make inferences, make sure all of it fits together without logical errors, etc.

These people you're talking about must be rare online, as human communication is pretty rife with logical errors.

Since that November in which this technology boomed we have been much too often reading "people also drink from puddles", as if it were standard practice.

That we implement skills, not deficiencies, is a basic concept that is getting to such a level of needed visibility it should probably be inserted in the guidelines.

We implement skills, not deficiencies.

Re: Is chain-of-thought AI reasoning a mirage?

#65

Post by mdp2021 »

gdbsjjdn wrote: post #62
empath75 wrote: post #14

Earlier quoted context omitted.

This is an obnoxious response. Of course I recognize that philosophy is the solution to this. What I am pointing out is that philosophy has not as of yet resolved these relatively new problems. The idea that non-human intelligences might exist is of course an old one, but that is different from having an actual (potentially) existing one to reckon with.

> Writings on metacognition date back at least as far as two works by the Greek philosopher Aristotle (384–322 BC): On the Soul and the Parva Naturalia We built a box that spits out natural language and tricks humans into believing it's conscious. The box itself actually isn't that interesting, but the human side of the equation is.

> the human side of the equation is

You have only proven the urgency of Intelligence, the need to produce it in inflationary amounts.

Re: Is chain-of-thought AI reasoning a mirage?

#66

Post by Enginerrrd »

phailhaus wrote: post #51
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute.

It's true that if it's not producing text, there is no thinking involved, but it is absolutely NOT clear that the attention block isn't holding state and modeling something as it works to produce text predictions. In fact, I can't think of a way to define it that would make that untrue... unless you mean that there isn't a system wherein something like attention is updating/computing and the model itself chooses when to make text predictions. That's by design, but what you're arguing doesn't really follow.

Now, whether what the model is thinking about inside that attention block matches up exactly or completely with the text it's producing as generated context is probably at least a little dubious, and its unlikely to be a complete representation regardless.

Re: Is chain-of-thought AI reasoning a mirage?

#67

Post by ofjcihen »

stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

It’s incredible to me that so many seem to have fallen for “humans are just LLMs bruh” argument but I think I’m beginning to understand the root of the issue.

People who only “deeply” study technology only have that frame of reference to view the world so they make the mistake of assuming everything must work that way, including humans.

If they had a wider frame of reference that included, for example, Early Childhood Development, they might have enough knowledge to think outside of this box and know just how ridiculous that argument is.

Re: Is chain-of-thought AI reasoning a mirage?

#68

Post by viccis »

adastra22 wrote: post #23
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

>As far as I can tell “generating context” is exactly what human reasoning is too.

This was the view of Hume (humans as bundles of experience who just collect information and make educated guesses for everything). Unfortunately, it leads to philosophical skepticism, in which you can't ground any knowledge absolutely, as it's all just justified by some knowledge you got from someone else, which also came from someone else, etc., and eventually you can't actually justify any knowledge that isn't directly a result of experience (the concept of "every effect has a cause" is a classic example).

There have been plenty of epistemological responses to this viewpoint, with Kant's view, of humans doing a mix of "gathering context" (using our senses) but also applying universal categorical reasoning to schematize and understand / reason from the objects we sense, being the most well known.

I feel like anyone talking about the epistemology of AI should spend some time reading the basics of all of the thought from the greatest thinkers on the subject in history...

Re: Is chain-of-thought AI reasoning a mirage?

#69

Post by brunokim »

I'm unconvinced by the article criticism's, given they also employ their feels and few citations.

> I appreciate that research has to be done on small models, but we know that reasoning is an emergent capability! (...) Even if you grant that what they’re measuring is reasoning, I am profoundly unconvinced that their results will generalize to a 1B, 10B or 100B model.

A fundamental part of applied research is simplifying a real-world phenomenon to better understand it. Dismissing that for this many parameters, for such a simple problem, the LLM can't perform out of distribution just because it's not big enough undermines the very value of independent research. Tomorrow another model with double the parameters may or may not show the same behavior, but that finding will be built on top of this one.

Also, how do _you_ know that reasoning is emergent, and not rationalising on top of a compressed version of the web stored in 100B parameters?

Post reply on HN