Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is chain-of-thought AI reasoning a mirage?

seangoedecke.com

Post reply on HN
Print view View original

111–120 of 191 posts

Re: Is chain-of-thought AI reasoning a mirage?

#111

Post by js8 »

viccis wrote: post #68
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

>As far as I can tell “generating context” is exactly what human reasoning is too. This was the view of Hume (humans as bundles of experience who just collect information and make educated guesses for everything). Unfortunately, it leads to philosophical skepticism, in which you can't ground any knowledge absolutely, as it's all just justified by some knowledge you got from someone else, which also came from someone…

> I feel like anyone talking about the epistemology of AI should spend some time reading the basics

I agree, I think the problem with AI is we don't know or haven't formalized enough what epistemology should AGI systems have. Instead, people are looking for shortcuts, feeding huge amount of data into the models, hoping it will self-organize into something that humans actually want.

Re: Is chain-of-thought AI reasoning a mirage?

#112

Post by griffzhowl »

modeless wrote: post #108
griffzhowl wrote: post #107

Earlier quoted context omitted.

To be more clear about why I disagree the cases are parallel: We know how a submarine moves through water, whether it's "swimming" isn't an interesting question. We don't know to what extent a machine can reproduce the cognitive functions of a human. There are substantive and significant questions about whether or to what extent a particular machine or program can reproduce human cognitive functions. So I might have…

"We know how it moves" is not the reason the question of whether a submarine swims is not interesting. It's because the question is mainly about the definition of the word "swim" rather than about capabilities. > if that's what we mean by the question whether a machine can think That's the issue. The question of whether a machine can think (or reason) is a question of word definitions, not capabilities. The capabilit…

> The capabilities questions are the ones that matter.

Yes, that's what I'm saying. I also think there's a clear sense in which asking whether machines can think is a question about capabilities, even though we would need a more precise definition of "thinking" to be able to answer it.

So that's how I'd sum it up: we know the capabilities of submarines, and whether we say they're swimming or not doesn't answer any further question about those capabilities. We don't know the capabilities of machines; the interesting questions are about what they can do, and one (imprecise) way of asking that question is whether they can think

Re: Is chain-of-thought AI reasoning a mirage?

#113

Post by viccis »

js8 wrote: post #111
viccis wrote: post #68

Earlier quoted context omitted.

>As far as I can tell “generating context” is exactly what human reasoning is too. This was the view of Hume (humans as bundles of experience who just collect information and make educated guesses for everything). Unfortunately, it leads to philosophical skepticism, in which you can't ground any knowledge absolutely, as it's all just justified by some knowledge you got from someone else, which also came from someone…

> I feel like anyone talking about the epistemology of AI should spend some time reading the basics I agree, I think the problem with AI is we don't know or haven't formalized enough what epistemology should AGI systems have. Instead, people are looking for shortcuts, feeding huge amount of data into the models, hoping it will self-organize into something that humans actually want.

It's partly driven by a hope that if you can model language well enough, you'll then have a model of knowledge. Logical positivism tried that with logical systems, which are much more precise languages of expressing facts, and it still fell on its face.

Re: Is chain-of-thought AI reasoning a mirage?

#114

Post by _diyar »

safety1st wrote: post #87
LudwigNagasena wrote: post #75

> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…

I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work? I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministica…

We expect this approach to work because it's currently the best working approach. Nothing else comes close.

Using symbolic language is a good idea in theory, but in practice it doesn't scale as well as auto-regression + RL.

The IMO results of DeepMind illustrate this well: In 2024, they solved it using AlphaProof and AlphaGeometry, using the Lean language as a formal symbolic logic[1]. In 2025 they performed better and faster by just using a fancy version of Gemini, only using natural language[2].

[1] https://deepmind.google/discover/blog/ai-solves-imo-problems...

[2] https://deepmind.google/discover/blog/advanced-version-of-ge...

Note: I agree with the notion of the parent comment that letting the models reason in latent space might make sense, but that's where I'm out of my depth.

Re: Is chain-of-thought AI reasoning a mirage?

#115

Post by modeless »

griffzhowl wrote: post #112
modeless wrote: post #108

Earlier quoted context omitted.

"We know how it moves" is not the reason the question of whether a submarine swims is not interesting. It's because the question is mainly about the definition of the word "swim" rather than about capabilities. > if that's what we mean by the question whether a machine can think That's the issue. The question of whether a machine can think (or reason) is a question of word definitions, not capabilities. The capabilit…

> The capabilities questions are the ones that matter. Yes, that's what I'm saying. I also think there's a clear sense in which asking whether machines can think is a question about capabilities, even though we would need a more precise definition of "thinking" to be able to answer it. So that's how I'd sum it up: we know the capabilities of submarines, and whether we say they're swimming or not doesn't answer any fu…

> I also think there's a clear sense in which asking whether machines can think is a question about capabilities, even though we would need a more precise definition of "thinking" to be able to answer it.

The second half of the sentence contradicts the first. It can't be a clear question about capabilities without widespread agreement on a more rigorous definition of the word "think". Dijkstra's point is that the debate about word definitions is irrelevant and a distraction. We can measure and judge capabilities directly.

Re: Is chain-of-thought AI reasoning a mirage?

#116

Post by griffzhowl »

modeless wrote: post #115
griffzhowl wrote: post #112

Earlier quoted context omitted.

> The capabilities questions are the ones that matter. Yes, that's what I'm saying. I also think there's a clear sense in which asking whether machines can think is a question about capabilities, even though we would need a more precise definition of "thinking" to be able to answer it. So that's how I'd sum it up: we know the capabilities of submarines, and whether we say they're swimming or not doesn't answer any fu…

> I also think there's a clear sense in which asking whether machines can think is a question about capabilities, even though we would need a more precise definition of "thinking" to be able to answer it. The second half of the sentence contradicts the first. It can't be a clear question about capabilities without widespread agreement on a more rigorous definition of the word "think". Dijkstra's point is that the deb…

> Dijkstra's point is that the debate about word definitions is irrelevant and a distraction.

Agreed, and I've made this point a few times, so it's ironic we're going back and forth about this.

> The second half of the sentence contradicts the first.

I'm not saying the question is clear. I'm saying there's clearly an interpretation of it as a question about capabilities.

Re: Is chain-of-thought AI reasoning a mirage?

#119

Post by adastra22 »

stonemetal12 wrote: post #39
adastra22 wrote: post #23

Earlier quoted context omitted.

You are making the same mistake OP is calling out. As far as I can tell “generating context” is exactly what human reasoning is too. Consider the phrase “let’s reason this out” where you then explore all options in detail, before pronouncing your judgement. Feels exactly like what the AI reasoner is doing.

"let's reason this out" is about gathering all the facts you need, not just noting down random words that are related. The map is not the terrain, words are not facts.

So when you say “that’s the reason this out” you open up Wikipedia or reference textbooks and start gathering facts? I mean that’s great, but I certainly don’t. Most of the time “gathering facts” means recalling relevant info from memory. Which is roughly what the LLM is doing, no?

Re: Is chain-of-thought AI reasoning a mirage?

#120

Post by energy123 »

kelipso wrote: post #56
energy123 wrote: post #52

Earlier quoted context omitted.

Performance is proportional to the number of reasoning tokens. How to reconcile that with your opinion that they are "random words"?

Technically random can have probabilities associated with them.. Casual speech, random means equal probabilities, or we don’t know the probabilities. But for LLM token output, it does estimate the probabilities.

Greedy decoding isn't random.
Post reply on HN