Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

91–100 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#91

Post by llllm »

kazinator wrote: post #80
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

Not all reasoning requires language. Symbolic reasoning uses language. Real-time spatial reasoning like driving a car and not hitting things does not seem linguistic. Figuring out how to rotate a cabinet so that it will clear through a stairwell also doesn't seem like it requires language, only to communicate the solution to someone else (where language can turn into a hindrance, compared to a diagram or model).

Pivot!

Re: Is chain-of-thought AI reasoning a mirage?

#92

Post by emorning4 »

LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

>That is an unreasonable assumption.Neither language nor writing is required to perform logic, see (Existential graph)[https://en.wikipedia.org/wiki/Existential_graph].

You can perform automated logic by putting/removing beads on a colored tile floor, see (Flower Calculus)[https://arxiv.org/abs/2402.15174].

Re: Is chain-of-thought AI reasoning a mirage?

#93

Post by gond »

ofjcihen wrote: post #67
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

It’s incredible to me that so many seem to have fallen for “humans are just LLMs bruh” argument but I think I’m beginning to understand the root of the issue. People who only “deeply” study technology only have that frame of reference to view the world so they make the mistake of assuming everything must work that way, including humans. If they had a wider frame of reference that included, for example, Early Childhoo…

That is an issue prevalent in the western world for the last 200 years, beginning possibly with the Industrial Revolution, probably earlier. That problem is reductionism, consequently applied down to the last level: discover the smallest element of every field of science, develop an understanding of all the parts from the smallest part upwards and develop, from the understanding of the parts, an understanding of the whole.

Unfortunately, this approach does not yield understanding, it yields know-how.

Re: Is chain-of-thought AI reasoning a mirage?

#94

Post by dmacfour »

ofjcihen wrote: post #67
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

It’s incredible to me that so many seem to have fallen for “humans are just LLMs bruh” argument but I think I’m beginning to understand the root of the issue. People who only “deeply” study technology only have that frame of reference to view the world so they make the mistake of assuming everything must work that way, including humans. If they had a wider frame of reference that included, for example, Early Childhoo…

I have a background in ML and work in software development, but studied experimental psych in a past life. It's actually kind of painful watching people slap phases related to cognition onto things that aren't even functionally equivalent to their namesakes, then parade them around like some kind of revelation. It's also a little surprising that there no interest (at least publicly) in using cognitive architectures in the development of AI systems.

Re: Is chain-of-thought AI reasoning a mirage?

#95

Post by jrm4 »

Current thought, for me there's a lot of hand-wringing about what is "reasoning" and what isn't. But right now perhaps the question might be boiled down to -- "is the bottleneck merely hard drive space/memory/computing speed?"

I kind of feel like we won't be able to even begin to test this until a few more "Moore's law" cycles.

Re: Is chain-of-thought AI reasoning a mirage?

#96

Post by shannifin »

safety1st wrote: post #87
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work? I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministica…

Problem is, even with symbolic logic, reasoning is not completely deterministic. Whether one can get to a set of given axioms from a given proposition is sometimes undecidable.

Re: Is chain-of-thought AI reasoning a mirage?

#97

Post by j45 »

Currently it feels like it's more simulated chain-of-thought / reasoning, sometimes very consistent, but simulated, partially because it's statistically generated and non-deterministic (not the exact same path to the similar or same each response run).

Re: Is chain-of-thought AI reasoning a mirage?

#98

Post by js8 »

I think LLM's chain of thought is reasoning. When trained, LLM sees lot of examples like "All men are mortal. Socrates is a man." followed by "Therefore, Socrates is mortal.". This causes the transformer to learn rule "All A are B. C is A." is often followed by "Therefore, C is B." And so it can apply this logical rule, predictively. (I have converted the example from latent space to human language for clarity.)

Unfortunately, sometimes LLM also learns "All A are C. All B are C." is followed by "Therefore, A is B.", due to bad example in the training data. (More insidiously, it might learn this rule only in a special case.)

So it learns some logic rules but not consistently. This lack of consistency will cause it to fail on larger problems.

I think NNs (transformers) could be great in heuristic suggesting which valid logical rules (could be even modal or fuzzy logic) to apply in order to solve a certain formalized problem, but not so great at coming up with the logic rules themselves. They could also be great at transforming the original problem/question from human language into some formal logic, that would then be resolved using heuristic search.

Re: Is chain-of-thought AI reasoning a mirage?

#99

Post by Vegenoid »

benreesman wrote: post #76
stonemetal12 wrote: post #20

When Using AI they say "Context is King". "Reasoning" models are using the AI to generate context. They are not reasoning in the sense of logic, or philosophy. Mirage, whatever you want to call it, it is rather unlike what people mean when they use the term reasoning. Calling it reasoning is up there with calling generating out put people don't like hallucinations.

People will go to extremely great lengths to debate the appropriate analogy for how these things work, which is fun I guess but in a "get high with a buddy" sense at least to my taste. Some of how they work is well understood (a lot now, actually), some of the outcomes are still surprising. But we debate both the well understood parts and the surprising parts both with the wrong terminology borrowed from pretty dubio…

While I largely agree with you, more abstract judgements must be made as the capabilities (and therefore tasks being completed) become increasingly general. Attempts to boil human intellectual capability down to "X performance on Y task according to Z eval" can be useful, but are famously incomplete and insufficient on their own for making good decisions about which humans (a.k.a. which general intelligences) are useful and how to utilize and improve them. Boiling down highly complex behavior into a small number of metrics loses a lot of detail.

There is also the desire to discover why a model that outperforms others does so, so that the successful technique can be refined and applied elsewhere. This too usually requires more approaches than metric comparison.

Re: Is chain-of-thought AI reasoning a mirage?

#100

Post by mrspuratic »

azrazalea_debt wrote: post #37
stonemetal12 wrote: post #24

Earlier quoted context omitted.

> it was tested on a 4 layer deep toy model How do you see that impacting the results? It is the same algorithm just on a smaller scale. I would assume a 4 layer model would not be very good, but does reasoning improve it? Is there a reason scale would impact the use of reasoning?

A lot of current LLM work is basically emergent behavior. They use a really simple core algorithm and scale it up, and interesting things happen. You can read some of anthropic's recent papers to see some of this, like: They didn't expect LLMs could "lookahead" when writing poetry. However, when they actually went in and watched what was happening (there's details on how this "watching" works on their blog/in their s…

On that topic, it seems backwards to me: intelligence is not emergent behaviour of language, rather the opposite.
Post reply on HN