Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

arstechnica.com

Post reply on HN
Print view View original

31–40 of 140 posts

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#31

Post by OtherShrezzing »

NitpickLawyer wrote: post #4

> Without specification, we employ a decoder-only language model GPT2 (Radford et al., 2019) with a configuration of 4 layers, 32 hidden dimensions, and 4 attention heads. Yeah, ok. The research is interesting, warranted, but writing an article about it, and leading with the conclusions gathered from toy models and implying this generalises to production LLMs is useless. We've been here before with small models. Trai…

I think it is worth writing about simply because it might get the (cost constrained) researcher’s work in front of someone who has the near-unlimited research budgets at one of the big AI companies.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#32

Post by XenophileJKO »

refulgentis wrote: post #22
willvarfar wrote: post #19

Earlier quoted context omitted.

I was just at the KDD conference and the general consensus agreed with this paper. There was only one keynoter who just made the assumption that LLMs are associated with reasoning, which was jarring as the previous keynoter had just explained at length why we need a neuro-symbolic approach instead. The thing is, I think the current companies making LLMs are _not_ trying to be correct or right. They are just trying to…

Not sure what all this is about, I somewhat regret taking a breaking from coding with LLMs to have it explained to me its all a mirage and a secret and sloppy plan for getting me an automagic egirl or something. ;)

Right? Oh this fairly novel solution the the problem I was having that works and is well tested. Oh throw it away.. sorry the model can't think of stuff..

Back to square one!!

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#33

Post by ricardobeat »

kazinator wrote: post #27
NitpickLawyer wrote: post #4

> Without specification, we employ a decoder-only language model GPT2 (Radford et al., 2019) with a configuration of 4 layers, 32 hidden dimensions, and 4 attention heads. Yeah, ok. The research is interesting, warranted, but writing an article about it, and leading with the conclusions gathered from toy models and implying this generalises to production LLMs is useless. We've been here before with small models. Trai…

> conclusions gathered from toy models and implying this generalises to production LLMs is useless You are just trotting out the tired argument that model size magically fixes the issues, rather than just improves the mirage, and so nothing can be known about models with M parameters by studying models with N Given enough parameters, a miraculous threshold is reached whereby LLMs switch from interpolating to extrapol…

That’s what has been seen in practice though. SOTA LLMs have been shown again and again to solve problems unseen in their data set; and despite their shortcomings they have become extremely useful for a wide variety of tasks.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#34

Post by bubblyworld »

syllogism wrote: post #16

It's interesting that there's still such a market for this sort of take. > In a recent pre-print paper, researchers from the University of Arizona summarize this existing work as "suggest[ing] that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text." What does this even mean? Let's veto the word "reasoning" here and reflect. The LLM produces a series of outputs. Each output c…

Not sure why everyone is downvoting you as I think you raise a good point - these anthropomorphic words like "reasoning" are useful as shorthands for describing patterns of behaviour, and are generally not meant to be direct comparisons to human cognition. But it goes both ways. You can still criticise the model on the grounds that what we call "reasoning" in the context of LLMs doesn't match the patterns we associate with human "reasoning" very well (such as ability to generalise to novel situations), which is what I think the authors are doing.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#35

Post by ceejayoz »

antihipocrat wrote: post #21
PeterStuer wrote: post #17

Earlier quoted context omitted.

Please don't tell me you were one of those marking every SO question as duplicate, more often than not missing the entire nuance in the question that made it not a duplicate at all, and the answers to the so called previously asked question utterly unusable? This was one of those infuriating things that drove so many away from SO and jump ship the second there was an alternative.

I'm not sure why duplicates were ever considered an issue. For certain subjects (like JS) things evolved so quickly during the height of SO that even a year old answer was outdated. That and search engines seemed to promote more recent content.. so an old answer sank under the ocean of blog spam

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki.

If a year-old answer on a canonical question is now incorrect, you edit it.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#36

Post by loosetypes »

ricardobeat wrote: post #33
kazinator wrote: post #27

Earlier quoted context omitted.

> conclusions gathered from toy models and implying this generalises to production LLMs is useless You are just trotting out the tired argument that model size magically fixes the issues, rather than just improves the mirage, and so nothing can be known about models with M parameters by studying models with N Given enough parameters, a miraculous threshold is reached whereby LLMs switch from interpolating to extrapol…

That’s what has been seen in practice though. SOTA LLMs have been shown again and again to solve problems unseen in their data set; and despite their shortcomings they have become extremely useful for a wide variety of tasks.

Mind linking any examples (or categories) of problems that are definitively not in pre training data but can still be solved by LLMs? Preferably something factual rather than creative, genuinely curious.

Dumb question but anything like this that’s written about on the internet will ultimately end up as training fodder, no?

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#38

Post by bsaul »

willvarfar wrote: post #19
NitpickLawyer wrote: post #14

Earlier quoted context omitted.

I have nothing against researching this, I think it's important. My main issue is with articles choosing to grab a "conclusion" and imply it extrapolates to larger models, without any support for that. They are going for the catchy title first, fine-print be damned.

I was just at the KDD conference and the general consensus agreed with this paper. There was only one keynoter who just made the assumption that LLMs are associated with reasoning, which was jarring as the previous keynoter had just explained at length why we need a neuro-symbolic approach instead. The thing is, I think the current companies making LLMs are _not_ trying to be correct or right. They are just trying to…

"as the previous keynoter had just explained at length why we need a neuro-symbolic approach instead"

Do you have a link to the video for that talk ?

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#39

Post by jongjong »

I've used LLMs to generate code for a custom serverless framework which I wrote from scratch that it had never seen before. The framework follows some industry conventions but applied in a distinct way with some distinct features which I have not yet encountered in any other framework...

I'm willing to accept that maybe LLMs cannot invent entirely new concepts but I know for a fact that they can synthesize and merge different unfamiliar concepts in complex logical ways to deliver new capabilities. This is valuable on its own.

Re: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

#40

Post by MoreQARespect »

ceejayoz wrote: post #35
antihipocrat wrote: post #21

Earlier quoted context omitted.

I'm not sure why duplicates were ever considered an issue. For certain subjects (like JS) things evolved so quickly during the height of SO that even a year old answer was outdated. That and search engines seemed to promote more recent content.. so an old answer sank under the ocean of blog spam

SO wanted to avoid being a raw Q&A site in favor of something more like a wiki. If a year-old answer on a canonical question is now incorrect, you edit it .

That's a valid goal, but they should have adapted the software to the community instead of trying to adapt the community to the software.

SO's biggest asset was its community and while they treated it with some respect in the beginning they took it for granted and trashed it later.

Post reply on HN