Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Reasoning LLMs

magazine.sebastianraschka.com

Post reply on HN
Print view View original

131–140 of 196 posts

Re: Understanding Reasoning LLMs

#131

Post by goatlover »

nicce wrote: post #123
omnicognate wrote: post #122

Earlier quoted context omitted.

I doubt Dijkstra was unable to distinguish between a submarine and a swimmer.

The end result here is to move in the water. Both swimmer and submarine can do that. Whether submarine can swim like human, is irrelevant.

It's relevant if the claim is stronger than the submarine moves in water. If instead one were to say the submarine mimics human swimming, that would be false. Which is what we often see with claims regarding AGI.

In that regard, it's a bit of a false analogy, because submarines were never meant to mimic human swimming. But AI development often has that motivation. We could just say we're developing powerful intelligence amplification tools for use by humans, but for whatever reason, everyone prefers the scifi version. Augumented Intelligence is the forgotten meaning of AI.

Submarines never replaced human swimming (we're not whales), they enabled human movement under water in a way that wasn't possible before.

Re: Understanding Reasoning LLMs

#132

Post by wolfgangK »

kavalg wrote: post #111
mitthrowaway2 wrote: post #72

Earlier quoted context omitted.

I think this is because they're trained using RL, and math and coding problems offer an easy way to automatically assess an answer's correctness. I'm not sure how you'd score the correctness of other types of reasoning problems without a lot of manual (and highly subjective!) effort. Perhaps using simulations and games?

but even then it is not so trivial. Yesterday I gave DeepSeek a simple diophantine equation and it got it wrong 3 times, tried to correct itself and didn't end on a correct solution, but rather lied that the final solution is correct.

DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?

Re: Understanding Reasoning LLMs

#133

Post by agentultra »

sigbottle wrote: post #61

One thing I don't like about the trend in reasoning LLMs is the over-optimization to coding problems / math problems in particular. A lot of things that aren't well-defined require reasoning, and not just in a "SWE is ambiguous" kind of way - for example, thinking about how to present/teach something in a good way, iterating with the learner, thinking about what context they could be missing, etc. I find that all of…

Humans and other animals with cognition have the ability to form theories about the minds of others and can anticipate their reactions.

I don’t know if vector spaces and transformers can encode that ability.

It’s a key skill in thinking and writing. I definitely tailor my writing for my audience in order to get a point across. Often the goal isn't simply an answer, it’s a convincing answer.

Update: forgot a word

Re: Understanding Reasoning LLMs

#134

Post by voxic11 »

godelski wrote: post #119
ogrisel wrote: post #109

Earlier quoted context omitted.

Software Engineering is difficult to verify because it requires dealing with ambiguous understanding of the end-user actual needs / value and subtle trade-offs about code maintainability vs feature coverage vs computational performance. Algorithmic puzzles, on the other hand, both require reasoning and are easy to verify. There are other things in coding that are both useful and easy to verify: checking that the gene…

I agree with you on the first part, but no, code is not easy to verify. I think you missed part of what I wrote. I mean verify that your code is bug free. This cannot be done purely through testing. Formal verification still remains an unsolved problem.

Formal verification of arbitrary programs with arbitrary specifications will remain an unsolved problem (see halting problem). But formal verification of specific programs with specific specifications definitely is a solved problem.

Re: Understanding Reasoning LLMs

#135

Post by aldanor »

Dansvidania wrote: post #129

Are reasoning models -basically- generating their own context? as in, if a user were to feed prompt + those reasoning tokens as a prompt to a non-reasoning model, would the effect be functionally similar? I am sure this is improperly worded, I apologise.

Yes, more or less. Just like any LLM "generates its own context", during inference it doesn't care where the previous tokens came from. Inference doesn't have to change much, it's the training process that's different.

Re: Understanding Reasoning LLMs

#136

Post by seydor »

lysecret wrote: post #130

I think the next big problem we will run into with these line of reasoning models is "over-thinking" you can already start to see it. Thinking harder is not the universal pareto improvement everyone seems to think it is. (I understand the irony of using think 4 times here haha)

Reasoning is about serially applying a set of premises over and over to come to conclusions. But some of our biggest problems require thinking outside the box, sometimes way outside it, and a few times ingeniously making up a whole new set of premises, seemingly ex nihilo (or via divine inspiration). We are still in very early stages of making thinking machines.

Re: Understanding Reasoning LLMs

#137

Post by tpswa »

lysecret wrote: post #130

I think the next big problem we will run into with these line of reasoning models is "over-thinking" you can already start to see it. Thinking harder is not the universal pareto improvement everyone seems to think it is. (I understand the irony of using think 4 times here haha)

This is a natural next area of research. Nailing "adaptive compute" implies figuring out which problems to use more compute on, but I imagine this will get better as the RL does.

Re: Understanding Reasoning LLMs

#138

Post by ngneer »

cwillu wrote: post #25
ngneer wrote: post #19

Nice article. >Whether and how an LLM actually "thinks" is a separate discussion. The "whether" is hardly a discussion at all. Or, at least one that was settled long ago. "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." --Edsger Dijkstra

The document that quote comes from is hardly a definitive discussion of the topic. “[…] it tends to divert the research effort into directions in which science can not—and hence should not try to—contribute.” is a pretty myopic take. -- http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD898.PDF

Dijkstra myopic. Got it.

Re: Understanding Reasoning LLMs

#139

Post by soulofmischief »

agentultra wrote: post #133
sigbottle wrote: post #61

One thing I don't like about the trend in reasoning LLMs is the over-optimization to coding problems / math problems in particular. A lot of things that aren't well-defined require reasoning, and not just in a "SWE is ambiguous" kind of way - for example, thinking about how to present/teach something in a good way, iterating with the learner, thinking about what context they could be missing, etc. I find that all of…

Humans and other animals with cognition have the ability to form theories about the minds of others and can anticipate their reactions. I don’t know if vector spaces and transformers can encode that ability. It’s a key skill in thinking and writing. I definitely tailor my writing for my audience in order to get a point across. Often the goal isn't simply an answer, it’s a convincing answer. Update : forgot a word

What we do with the vectors is important, but vectors literally just hold information, I don't know how you can possibly strike out the possibility of advanced intelligence just because of the logical storage medium.

Re: Understanding Reasoning LLMs

#140

Post by soulofmischief »

bglazer wrote: post #83
mitthrowaway2 wrote: post #72

Earlier quoted context omitted.

I think this is because they're trained using RL, and math and coding problems offer an easy way to automatically assess an answer's correctness. I'm not sure how you'd score the correctness of other types of reasoning problems without a lot of manual (and highly subjective!) effort. Perhaps using simulations and games?

Games seem like a really under-explored source of data. It’s an area where humans have an intrinsic motivation to interact with others in dialogue, they can be almost arbitrarily open ended, and there tends to be the kind of clean success/failure end states that RL needs. I’m reminded of the high skill Diplomacy bot that Facebook research built but hasn’t really followed up on.

People are definitely trying to bridge the gap. https://deepmind.google/discover/blog/genie-2-a-large-scale-...
Post reply on HN