Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Reasoning LLMs

magazine.sebastianraschka.com

Post reply on HN
Print view View original

111–120 of 196 posts

Re: Understanding Reasoning LLMs

#111

Post by kavalg »

mitthrowaway2 wrote: post #72
sigbottle wrote: post #61

One thing I don't like about the trend in reasoning LLMs is the over-optimization to coding problems / math problems in particular. A lot of things that aren't well-defined require reasoning, and not just in a "SWE is ambiguous" kind of way - for example, thinking about how to present/teach something in a good way, iterating with the learner, thinking about what context they could be missing, etc. I find that all of…

I think this is because they're trained using RL, and math and coding problems offer an easy way to automatically assess an answer's correctness. I'm not sure how you'd score the correctness of other types of reasoning problems without a lot of manual (and highly subjective!) effort. Perhaps using simulations and games?

but even then it is not so trivial. Yesterday I gave DeepSeek a simple diophantine equation and it got it wrong 3 times, tried to correct itself and didn't end on a correct solution, but rather lied that the final solution is correct.

Re: Understanding Reasoning LLMs

#112

Post by moffkalast »

sigbottle wrote: post #61

One thing I don't like about the trend in reasoning LLMs is the over-optimization to coding problems / math problems in particular. A lot of things that aren't well-defined require reasoning, and not just in a "SWE is ambiguous" kind of way - for example, thinking about how to present/teach something in a good way, iterating with the learner, thinking about what context they could be missing, etc. I find that all of…

> things that aren't well-defined

If it's not well defined then you can't do RL on it because without a clear cut reward function the model will learn to do some nonsense instead, simple as.

Re: Understanding Reasoning LLMs

#113

Post by goingcrazythro »

I was having a look at the DeepSeek-R1 technical report and found the "aha moment" claims quite smelly, given that they do not disclose if the base model contains any chain of thought or reasoning data.

However, we know the base model is DeepSeek V3. From the DeepSeek V3 technical report, paragraph in 5.1. Supervised Fine-Tuning:

> Reasoning Data. For reasoning-related datasets, including those focused on mathematics, code competition problems, and logic puzzles, we generate the data by leveraging an internal DeepSeek-R1 model. Specifically, while the R1-generated data demonstrates strong accuracy, it suffers from issues such as overthinking, poor formatting, and excessive length. Our objective is to balance the high accuracy of R1-generated reasoning data and the clarity and conciseness of regularly formatted reasoning data.

In 5.4.1 they also talk about some ablation experiment by not using the "internal DeepSeek-R1" generated data.

While the "internal DeepSeek-R1" model is not explained, I would assume this is a DeepSeek V2 or V2.5 tuned for chain of thought. Therefore, it seems to me the "aha moment" is just promoting the behaviour that was already present in V3.

In the "Self-evolution Process of DeepSeek-R1-Zero"/ Figure 3 they claim reinforcement learning also leads to the model generating longer CoT sequences, but again, this comes from V3, they even mention the fine tuning with "internal R1" led to "excessive length".

None of the blogpost, news, articles I have read explaining or commenting on DeepSeek R1 takes this into account. The community is scrambling to re-implement the pipeline (see open-r1).

At this point, I feel like I took a crazy pill. Am I interpreting this completely wrong? Can someone shed some light on this?

Re: Understanding Reasoning LLMs

#114

Post by mike_hearn »

It's curious that the models switch between languages and that has to be trained out of them. I guess the ablations have been done already, but it makes me wonder if they do this because it's somehow easier to do some parts of the reasoning on languages other than English and maybe they should be just allowed to get on with it?

Re: Understanding Reasoning LLMs

#115

Post by tmnvdb »

gibsonf1 wrote: post #47
tmnvdb wrote: post #28

Earlier quoted context omitted.

"There are no LLMS that reason" is a claim about language, namely that the word 'reason' can only ever be applied to humans.

Not at all, we are building conceptual reasoning machines, but it is an entirely different technology than GPT/LLM dl/ml etc. [1] [1] https://graphmetrix.com/trinpod-server

Conceptual reasoning machines rely on concrete, explicit and intelligble concepts and rules. People like this because it 'looks' like reasoning on the inside.

However, our brains, like language models, rely on implicit, distributed representations of concepts and rules.

So the intelligble representations of conceptual reasoning machines are maybe too strong a requirement for 'reasoning' unless you want to exclude humans too.

Re: Understanding Reasoning LLMs

#116

Post by nvtop »

goingcrazythro wrote: post #113

I was having a look at the DeepSeek-R1 technical report and found the "aha moment" claims quite smelly, given that they do not disclose if the base model contains any chain of thought or reasoning data. However, we know the base model is DeepSeek V3. From the DeepSeek V3 technical report, paragraph in 5.1. Supervised Fine-Tuning: > Reasoning Data. For reasoning-related datasets, including those focused on mathematics…

I'm also very skeptical of the significance of this "aha moment". Even if they didn't include chain-of-thoughts to the base model's training data (unlikely), there are still plenty of it on the modern Internet. OpenAI released 800k of reasoning steps which are publicly available, github repositories, examples in CoT papers... It's definitely not a novel concept for a model, that it somehow discovered by its own.

Re: Understanding Reasoning LLMs

#117

Post by tmnvdb »

goingcrazythro wrote: post #113

I was having a look at the DeepSeek-R1 technical report and found the "aha moment" claims quite smelly, given that they do not disclose if the base model contains any chain of thought or reasoning data. However, we know the base model is DeepSeek V3. From the DeepSeek V3 technical report, paragraph in 5.1. Supervised Fine-Tuning: > Reasoning Data. For reasoning-related datasets, including those focused on mathematics…

https://oatllm.notion.site/oat-zero

Re: Understanding Reasoning LLMs

#118

Post by tmnvdb »

daxfohl wrote: post #87

But how on earth do you train it? With regular LLMs, you get feedback on each word / token you generate, as you can match against training text. With these, you've got to generate hundreds of tokens in the thinking block fiest, and even after that, there's no "matching" next word, only a full solution. And it's either right or wrong, no probabilities to do a gradient on.

Only the answer is taken into account for scoring. The part is not.

Re: Understanding Reasoning LLMs

#119

Post by godelski »

ogrisel wrote: post #109
godelski wrote: post #107

Earlier quoted context omitted.

This is a misconception. Coding is very difficult to verify, it's just that everyone takes a good enough approach. They check the output and if it looks good they move on. But you can't just test and check your way through problems. If this was true we wouldn't have bugs lol. I hear you, your test set didn't have enough coverage. Great! Allow me to introduce you to black swans.

Software Engineering is difficult to verify because it requires dealing with ambiguous understanding of the end-user actual needs / value and subtle trade-offs about code maintainability vs feature coverage vs computational performance. Algorithmic puzzles, on the other hand, both require reasoning and are easy to verify. There are other things in coding that are both useful and easy to verify: checking that the gene…

I agree with you on the first part, but no, code is not easy to verify. I think you missed part of what I wrote. I mean verify that your code is bug free. This cannot be done purely through testing. Formal verification still remains an unsolved problem.

Re: Understanding Reasoning LLMs

#120

Post by nicce »

root_axis wrote: post #99
ngneer wrote: post #19

Nice article. >Whether and how an LLM actually "thinks" is a separate discussion. The "whether" is hardly a discussion at all. Or, at least one that was settled long ago. "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." --Edsger Dijkstra

That doesn't really settle it, just dismiss the question. The submarine analogy could be interpreted to support either conclusion.

Wasn’t the point that process does not matter if we can’t distinguish the end results?
Post reply on HN