Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reasoning models reason well, until they don't

arxiv.org

Post reply on HN
Print view View original

191–200 of 224 posts

Re: Reasoning models reason well, until they don't

#191

Post by daveguy »

dymk wrote: post #186
adastra22 wrote: post #150

Earlier quoted context omitted.

They are next token predictors though. That is literally wha they are. Nobody is saying they are simple Markov chains.

It’s a uselessly reductive statement. A person at a keyboard is also a next token predictor, then.

They are both designed, trained, and evaluated by how well they can predict the next token. It's literally what they do. "Reasoning" models just buildup additional context of next token predictions and RL is used to bias output options to ones more appealing to human judges. It's not a meme. It's an accurate description of their fundamental computational nature.

Re: Reasoning models reason well, until they don't

#193

Post by [deleted] »

HarHarVeryFunny wrote: post #189
gpderetta wrote: post #151

Earlier quoted context omitted.

Indeed, they are next token predictors, but this is a vacuous statement because the predictor can be arbitrary complex.

Sure, but a complex predictor is still a predictor. It would be a BAD predictor if everything it output was not based on "what would the training data say?". If you ask it to innovate and come up with something not in it's training data, what do you think it will do .... it'll "look at" it's training data and regurgitate (predict) something labelled as innovative You can put a reasoning cap on a predictor, but it's s…

[deleted]

Re: Reasoning models reason well, until they don't

#194

Post by astrange »

amelius wrote: post #95
My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

The problem is that the training data doesn't contain a lot of "I don't know".

That's not important compared to the post-training RL, which isn't "training data".

Re: Reasoning models reason well, until they don't

#195

Post by sothatsit »

riku_iki wrote: post #174
sothatsit wrote: post #6

Earlier quoted context omitted.

What do you mean by reasoning? If you mean solving logic problems, then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. Reasoning LLMs can also complete problems like multiplying large numbers, which requires applying some sort of algorithm where the results cannot just be memorised. They also do this much better than standard pre-trained LLMs with no RL. So, that makes m…

> then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. it could be this is just result of good stochastic parroting and not reasoning. Both of those niches are narrow with high amount of training data (e.g. corps buying solutions from leetcode and training LLMs on them). From another hand we see that LLMs fail in more complex environment: e.g. ask to build some new featur…

This is clearly false. LLMs being able to multiply large numbers is the clear example to me that there is more than just memorisation going on. They cannot just memorise the answers to multipling huge numbers like they do.

That's not to mention that these programming competition problems are designed to be novel. They are as novel as the competition designers can get while sticking to the bounds of the competition. This is clearly not stochastic parrot behaviour.

Additionally, them falling over in large codebases is not evidence that they cannot reason over smaller well-defined problems. It is just evidence that their reasoning has limits, which should not be surprising to anyone. Humans also have limits in our reasoning. That does not mean we do not reason.

Re: Reasoning models reason well, until they don't

#196

Post by riku_iki »

sothatsit wrote: post #195
riku_iki wrote: post #174

Earlier quoted context omitted.

> then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. it could be this is just result of good stochastic parroting and not reasoning. Both of those niches are narrow with high amount of training data (e.g. corps buying solutions from leetcode and training LLMs on them). From another hand we see that LLMs fail in more complex environment: e.g. ask to build some new featur…

This is clearly false. LLMs being able to multiply large numbers is the clear example to me that there is more than just memorisation going on. They cannot just memorise the answers to multipling huge numbers like they do. That's not to mention that these programming competition problems are designed to be novel. They are as novel as the competition designers can get while sticking to the bounds of the competition. T…

I think you just made lots of handwaving statements. Here is result which says LLMs can't do multi-digit multiplications well: https://arxiv.org/pdf/2510.00184

Re: Reasoning models reason well, until they don't

#197

Post by nl »

brap wrote: post #155
nl wrote: post #69

Earlier quoted context omitted.

This is where you get the natural language tool to write the formal logic. ChatGPT knows WebPPL really well for example.

You will need a formal language first. Take this statement for example: >ChatGPT knows WebPPL really well What formal language can express this statement? What will the text be parsed into? Which transformations can you use to produce other truthful (and interesting) statements from it? Is this flexible enough to capture everything that can be expressed in English? The closest that comes to mind is Prolog, but it doe…

> You will need a formal language first.

No, that's the entire point!

The LLM is the bridge between natural language and a formal specification.

(WebPPL is a formal language btw. It's not unlike Prolog but is designed from the start to express lemmas probabilistically)

Re: Reasoning models reason well, until they don't

#198

Post by sothatsit »

riku_iki wrote: post #196
sothatsit wrote: post #195

Earlier quoted context omitted.

This is clearly false. LLMs being able to multiply large numbers is the clear example to me that there is more than just memorisation going on. They cannot just memorise the answers to multipling huge numbers like they do. That's not to mention that these programming competition problems are designed to be novel. They are as novel as the competition designers can get while sticking to the bounds of the competition. T…

I think you just made lots of handwaving statements. Here is result which says LLMs can't do multi-digit multiplications well: https://arxiv.org/pdf/2510.00184

We are talking about reasoning models here, not old non-reasoning models like Llama-90B and GPT-4. Obviously, they cannot multiply numbers. That was never in question.

Maybe at least give a cursory glance at a paper before trying to cite it to support your point?

I find it fun that this paper also points out that using another training method, IcoT, they can produce models that can multiply numbers perfectly. The frontier reasoning models can still make mistakes, they just get very close a lot of the time, even with 10-20 digit numbers. But the IcoT models can do it perfectly, they just can only multiply numbers.

Re: Reasoning models reason well, until they don't

#199

Post by riku_iki »

sothatsit wrote: post #198
riku_iki wrote: post #196

Earlier quoted context omitted.

I think you just made lots of handwaving statements. Here is result which says LLMs can't do multi-digit multiplications well: https://arxiv.org/pdf/2510.00184

We are talking about reasoning models here, not old non-reasoning models like Llama-90B and GPT-4. Obviously, they cannot multiply numbers. That was never in question. Maybe at least give a cursory glance at a paper before trying to cite it to support your point? I find it fun that this paper also points out that using another training method, IcoT, they can produce models that can multiply numbers perfectly. The fro…

so, give ref on results which prove that they reliably can multiply arbitrary numbers

> Maybe at least give a cursory glance at a paper before trying to cite it to support your invalid point?

they use CoT aka reasoning steps

Re: Reasoning models reason well, until they don't

#200

Post by sothatsit »

riku_iki wrote: post #199
sothatsit wrote: post #198

Earlier quoted context omitted.

We are talking about reasoning models here, not old non-reasoning models like Llama-90B and GPT-4. Obviously, they cannot multiply numbers. That was never in question. Maybe at least give a cursory glance at a paper before trying to cite it to support your point? I find it fun that this paper also points out that using another training method, IcoT, they can produce models that can multiply numbers perfectly. The fro…

so, give ref on results which prove that they reliably can multiply arbitrary numbers > Maybe at least give a cursory glance at a paper before trying to cite it to support your invalid point? they use CoT aka reasoning steps

They do not apply reinforcement learning, which is what most people mean when they talk about reasoning LLMs. This means it is not comparable to the frontier reasoning models.

Here is the post I remember seeing: https://www.reddit.com/r/singularity/comments/1ip3vpa/multid...

This shows that o3-mini has >90% accuracy at multiplying numbers up to 8-digits, and it is capable of multiplying numbers much larger than that. Whereas, gpt-4o could only multiply 2-digit numbers reliably.

Here's another person I saw experimenting with this: https://sanand0.github.io/llmmath/

That's not to mention that if you give these models a Python interpreter, they can also do this task perfectly and tackle much more complicated tasks as well. Although, that is rather separate to the models themselves being able to apply the reasoning steps to multiply numbers.

Post reply on HN