Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

arstechnica.com

Post reply on HN
Print view View original

41–50 of 140 posts

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#41

Post by ceejayoz »

MoreQARespect wrote: post #40
ceejayoz wrote: post #35

Earlier quoted context omitted.

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki. If a year-old answer on a canonical question is now incorrect, you edit it .

That's a valid goal, but they should have adapted the software to the community instead of trying to adapt the community to the software. SO's biggest asset was its community and while they treated it with some respect in the beginning they took it for granted and trashed it later.

I think this policy was, in large part, intended to respect the user base, who get exhausted answering the same question over and over.

I do agree they later trashed that relationship with the Monica incident and AI policies.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#42

Post by moi2388 »

“ the researchers created a carefully controlled LLM environment in an attempt to measure just how well chain-of-thought reasoning works when presented with "out of domain" logical problems that don't match the specific logical patterns found in their training data.”

Why? If it’s out of domain we know it’ll fail.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#43

Post by PeterStuer »

ceejayoz wrote: post #35
antihipocrat wrote: post #21

Earlier quoted context omitted.

I'm not sure why duplicates were ever considered an issue. For certain subjects (like JS) things evolved so quickly during the height of SO that even a year old answer was outdated. That and search engines seemed to promote more recent content.. so an old answer sank under the ocean of blog spam

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki. If a year-old answer on a canonical question is now incorrect, you edit it .

But the answer has not become incorrect. It is still correct for that question in that specific context. More likely, the 'canonicalization process' was overly coarse (for SEO?), inconsistent and confused.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#44

Post by moi2388 »

ceejayoz wrote: post #35
antihipocrat wrote: post #21

Earlier quoted context omitted.

I'm not sure why duplicates were ever considered an issue. For certain subjects (like JS) things evolved so quickly during the height of SO that even a year old answer was outdated. That and search engines seemed to promote more recent content.. so an old answer sank under the ocean of blog spam

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki. If a year-old answer on a canonical question is now incorrect, you edit it .

Then they should have made a wiki instead of a Q&A site

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#45

Post by logicchains »

willvarfar wrote: post #10
NitpickLawyer wrote: post #4

> Without specification, we employ a decoder-only language model GPT2 (Radford et al., 2019) with a configuration of 4 layers, 32 hidden dimensions, and 4 attention heads. Yeah, ok. The research is interesting, warranted, but writing an article about it, and leading with the conclusions gathered from toy models and implying this generalises to production LLMs is useless. We've been here before with small models. Trai…

Doing analysis on small models or small data is perfectly valid if the results extrapolate to large models. Which is why right now we're looking at new research papers that are still listing the same small datasets and comparing to the same small models that papers five years ago did.

The extrapolation doesn't work if the transformer is too shallow (too few layers) relative to sequence length, because of https://arxiv.org/abs/2503.03961 . A bunch of tasks become unfeasible when the layer count is too low, and 4 layers is way too low. I.e. linearly increasing the number of layers in a model can result in a superlinear increase in performance on tasks like reasoning.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#46

Post by [deleted] »

syllogism wrote: post #16

It's interesting that there's still such a market for this sort of take. > In a recent pre-print paper, researchers from the University of Arizona summarize this existing work as "suggest[ing] that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text." What does this even mean? Let's veto the word "reasoning" here and reflect. The LLM produces a series of outputs. Each output c…

[deleted]

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#47

Post by Octoth0rpe »

moi2388 wrote: post #42

“ the researchers created a carefully controlled LLM environment in an attempt to measure just how well chain-of-thought reasoning works when presented with "out of domain" logical problems that don't match the specific logical patterns found in their training data.” Why? If it’s out of domain we know it’ll fail.

I don't think we know that it'll fail, or at least that is not universally accepted as true. Rather, there are claims that given a large enough model / context window, such capabilities emerge. I think skepticism of that claim is warranted. This research validates that skepticism, at least for a certain parameters (model family/size, context size, etc).

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#48

Post by wzdd »

syllogism wrote: post #16

It's interesting that there's still such a market for this sort of take. > In a recent pre-print paper, researchers from the University of Arizona summarize this existing work as "suggest[ing] that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text." What does this even mean? Let's veto the word "reasoning" here and reflect. The LLM produces a series of outputs. Each output c…

> The mechanism that the model uses to transition towards the answer is to generate intermediate text.

Yes, which makes sense, because if there's a landscape of states that the model is traversing, and there are probablistically likely pathways between an initial state and the desired output, but there isn't a direct pathway, then training the the model to generate intermediate text in order to move across that landscape so it can reach the desired output state is a good idea.

Presumably LLM companies are aware that there is (in general) no relationship between the generated intermediate text and the output, and the point of the article is that by calling it a "chain of thought" rather than "essentially-meaningless intermediate text which increases the number of potential states the model can reach" users are misled into thinking that the model is reasoning, and may then make unwarranted assumptions, such as that the model could in general apply the same reasoning to similar problems, which is in general not true.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#49

Post by exe34 »

kazinator wrote: post #30
NitpickLawyer wrote: post #14

Earlier quoted context omitted.

I have nothing against researching this, I think it's important. My main issue is with articles choosing to grab a "conclusion" and imply it extrapolates to larger models, without any support for that. They are going for the catchy title first, fine-print be damned.

Because model size is a trivial parameter, and not a new paradigm. What you're saying is like, you can't extrapolate that long division works on 100 digit numbers because you only worked through it using 7 digit numbers and a few small polynomials.

This is 100% how it doesn't work with LLMs.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#50

Post by ceejayoz »

moi2388 wrote: post #44
ceejayoz wrote: post #35

Earlier quoted context omitted.

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki. If a year-old answer on a canonical question is now incorrect, you edit it .

Then they should have made a wiki instead of a Q&A site

They did, really. That's why I can edit anyone else's questions and answers.
Post reply on HN