Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Reasoning LLMs

magazine.sebastianraschka.com

Post reply on HN
Print view View original

31–40 of 196 posts

Re: Understanding Reasoning LLMs

#31

Post by bloomingkales »

danielmarkbruce wrote: post #27
_yb2s wrote: post #22

Earlier quoted context omitted.

> they are incapable of even the simplest "out-of-distribution" deductive reasoning But the link demonstrates the opposite- these models absolutely are able to reason out of distribution, just not with perfect fidelity. The fact that they can do better than random is itself really impressive. And o1-preview does impressively well, only vary rarely getting the wrong answer on variants of that Alice in Wonderland probl…

anyone saying an LLM is a stochastic parrot doesn't understand them... they are just parroting what they heard.

There is definitely a mini cult of people that want to be very right about how everyone else is very wrong about AI.

Re: Understanding Reasoning LLMs

#32

Post by tmnvdb »

behnamoh wrote: post #2

doesn't it seem like these models are getting to the point where even conceiving their training and development is less and less possible for the general public? I mean, we already knew only a handful of companies with capital could train them, but at least the principles, algorithms, etc. were accessible to individuals who wanted to create their own - much simpler - models. it seems that era is quickly ending, and w…

We have been in the 'magic scaling' era for a while now. While the basic architecture of language models is reasonably simple and well understood, the emergent effects of making models bigger are largely magic even to the researchers, only to be studied emperically after the fact.

Re: Understanding Reasoning LLMs

#33

Post by danielmarkbruce »

bloomingkales wrote: post #31
danielmarkbruce wrote: post #27

Earlier quoted context omitted.

anyone saying an LLM is a stochastic parrot doesn't understand them... they are just parroting what they heard.

There is definitely a mini cult of people that want to be very right about how everyone else is very wrong about AI.

ie, the people that AI is dumb? Or you are saying I'm in a cult for being pro it - I'm definitely part of that cult - the "we already have agi and you have to contort yourself into a pretzel to believe otherwise" cult. Not sure if there is a leader though.

Re: Understanding Reasoning LLMs

#34

Post by truculent »

vector_spaces wrote: post #4

Is there any work being done in training LLMs on more restricted formal languages? Something like a constraint solver or automated theorem prover, but much lower level. Specifically something that isn't natural language. That's the only path I could see towards reasoning models being truly effective I know there is work being done with e.g. Lean integration with ChatGPT, but that's not what I mean exactly -- there's…

I think something like structured generation might work in this context

Re: Understanding Reasoning LLMs

#35

Post by _yb2s »

Jensson wrote: post #30
_yb2s wrote: post #22

Earlier quoted context omitted.

> they are incapable of even the simplest "out-of-distribution" deductive reasoning But the link demonstrates the opposite- these models absolutely are able to reason out of distribution, just not with perfect fidelity. The fact that they can do better than random is itself really impressive. And o1-preview does impressively well, only vary rarely getting the wrong answer on variants of that Alice in Wonderland probl…

> If you would listen to most of the people critical of LLMs saying they're a "stochastic parrot" - it should be impossible for them to do better than random on any out of distribution problem. Even just changing one number to create a novel math problem should totally stump them and result in entirely random outputs, but it does not. You don't seem to understand how they work, they recurse their solution meaning if…

> You don't seem to understand how they work

I don't think anyone understands how they work- these type of explanations aren't very complete or accurate. Such explanations/models allow one to reason out what types of things they should be capable of vs incapable of in principle regardless of scale or algorithm tweaks, and those predictions and arguments never match reality and require constant goal post shifting as the models are scaled up.

We understand how we brought them about via setting up an optimization problem in a specific way, that isn't the same at all as knowing how they work.

I tend to think in the totally abstract philosophical sense, independent of the type of model, at the limit of an increasingly capable function approximator trained on an increasingly large and diverse set of real world cause/effect time series data, you eventually develop and increasingly accurate and general predictive model of reality organically within the model. Some model types do have fundamental limits in their ability to scale like this, but we haven't yet found one with these models.

It is more appropriate to objectively test what they can and cannot do, and avoid trying to infer what we expect from how we think they work.

Re: Understanding Reasoning LLMs

#36

Post by bloomingkales »

danielmarkbruce wrote: post #33
bloomingkales wrote: post #31

Earlier quoted context omitted.

There is definitely a mini cult of people that want to be very right about how everyone else is very wrong about AI.

ie, the people that AI is dumb? Or you are saying I'm in a cult for being pro it - I'm definitely part of that cult - the "we already have agi and you have to contort yourself into a pretzel to believe otherwise" cult. Not sure if there is a leader though.

I didn't realize my post can be interpreted either way. I'll leave it ambiguous, hah. Place your bets I guess.

Re: Understanding Reasoning LLMs

#37

Post by o11c »

Legend2440 wrote: post #16
scarmig wrote: post #12

Earlier quoted context omitted.

> But I think it's inappropriate to claim that models like R1 are "good at deductive or inductive reasoning" when that is demonstrably not true, they are incapable of even the simplest "out-of-distribution" deductive reasoning: https://xcancel.com/JJitsev/status/1883158738661691878 Your link says that R1, not all models like R1, fails at generalization. Of particular note: > We expose DeepSeek R1 to the variations of…

The way the authors talk about LLMs really rubs me the wrong way. They spend more of the paper talking up the 'claims' about LLMs that they are going to debunk than actually doing any interesting study. They came into this with the assumption that LLMs are just a cheap trick. As a result, they deliberately searched for an example of failure, rather than trying to do an honest assessment of generalization capabilities…

What the hype crowd doesn't get is that for most people, "a tool that randomly breaks" is not useful.

Re: Understanding Reasoning LLMs

#38

Post by HarHarVeryFunny »

scarmig wrote: post #12
aithrowawaycomm wrote: post #8

I like Raschka's writing, even if he is considerably more optimistic about this tech than I am. But I think it's inappropriate to claim that models like R1 are "good at deductive or inductive reasoning" when that is demonstrably not true, they are incapable of even the simplest "out-of-distribution" deductive reasoning: https://xcancel.com/JJitsev/status/1883158738661691878 They are certainly capable of doing is a wi…

> But I think it's inappropriate to claim that models like R1 are "good at deductive or inductive reasoning" when that is demonstrably not true, they are incapable of even the simplest "out-of-distribution" deductive reasoning: https://xcancel.com/JJitsev/status/1883158738661691878 Your link says that R1, not all models like R1, fails at generalization. Of particular note: > We expose DeepSeek R1 to the variations of…

I'd expect that OpenAI's stronger reasoning models also don't generalize too far outside of the areas they are trained for. At the end of the day these are still just LLMs, trying to predict continuations, and how well they do is going to depend on how well the problem at hand matches their training data.

Perhaps the type of RL used to train them also has an effect on generalization, but choice of training data has to play a large part.

Re: Understanding Reasoning LLMs

#39

Post by HarHarVeryFunny »

gsam wrote: post #6
vector_spaces wrote: post #4

Is there any work being done in training LLMs on more restricted formal languages? Something like a constraint solver or automated theorem prover, but much lower level. Specifically something that isn't natural language. That's the only path I could see towards reasoning models being truly effective I know there is work being done with e.g. Lean integration with ChatGPT, but that's not what I mean exactly -- there's…

In my mind, the pure reinforcement learning approach of DeepSeek is the most practical way to do this. Essentially it needs to continually refine and find more sound(?) subspaces of the latent (embedding) space. Now this could be the subspace which is just Python code (or some other human-invented subspace), but I don't think that would be optimal for the overall architecture. The reason why it seems the most reasona…

DeepSeek's approach with R1 wasn't pure RL - they used RL only to develop R0 from their V3 base model, but then went though two iterations of using current model to generate synthetic reasoning data, SFT on that, then RL fine-tuning, and repeat.

Re: Understanding Reasoning LLMs

#40

Post by daveguy »

Legend2440 wrote: post #24
suddenlybananas wrote: post #23

Earlier quoted context omitted.

>They came into this with the assumption that LLMs are just a cheap trick. As a result, they deliberately searched for an example of failure, rather than trying to do an honest assessment of generalization capabilities. And lo and behold, they still found a glaring failure. You can't fault them for not buying into the hype.

But it is still dishonest to declare reasoning LLMs a scam simply because you searched for a failure mode. If given a few hundred tries, I bet I could find an example where you reason poorly too. Wikipedia has a whole list of common failure modes of human reasoning: https://en.wikipedia.org/wiki/List_of_fallacies

Well, given the success rate is no more than 90% in the best cases. You could probably find a failure in about 10 tries. The only exception is o1-preview. And this is just a simple substitution of parameters.
Post reply on HN