It’s because they generate a seeming of reasoning, and don’t actually reason! (Slams the door angrily) (stomps out angrily) (touches the grass angrily)
What is to reason, if not to generate a seeming of reasoning?
(tips fedora)
Reasoning models reason well, until they don't
21–30 of 224 posts
Re: Reasoning models reason well, until they don't
#22But I also fail catastrophically once a reasoning problem exceeds modest complexity.
But you recognise you are likely to fail and thus dont respond or redirect the problem to someone who has a greater likelihood of not failing.
If that were true, we would live in a utopia. People vote/legislate/govern/live/raise/teach/preach without ever learning to reason correctly.
Re: Reasoning models reason well, until they don't
#23The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from some source node to a target node.
My main critique is that I don't think there's evidence that this issue would persist after continuing to scale models to be larger and doing more RL. With a harness like what coding agents do these days and with sufficient tool use, I bet models could go much further on that reasoning benchmark. Otherwise, if the reasoning problem were entirely done within a single context window, it's expected that a complex enough reasoning problem would be too difficult for the model to solve.
Re: Reasoning models reason well, until they don't
#24But I also fail catastrophically once a reasoning problem exceeds modest complexity.
I also fail catastrophically when trying to push nails through walls by I expect my hammer to do better.
I have one hammer and I expect it to work on every nail and screw. If it's not a general hammer, what good is it now?
Re: Reasoning models reason well, until they don't
#25Earlier quoted context omitted.
What do you mean by reasoning? If you mean solving logic problems, then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. Reasoning LLMs can also complete problems like multiplying large numbers, which requires applying some sort of algorithm where the results cannot just be memorised. They also do this much better than standard pre-trained LLMs with no RL. So, that makes m…
I guess we mean here ”usefull reasoning” instead of the idiot-savant. I mean it’s a fair ask since these are marketed as _tools_ you can use to implement _industrial processes_ and even replace you human workers. In that I guess the model does not need to be the most reasonable intepreter of vague and poorly formulated user inputs but I think to improve a bit at least, to become usefull general appliances and not jus…
^^This is a great view and it seems generally widely understood by the file and rank techies. I feel pitty for the general public retail investors which are about to be left holding the bag for the VCs, after a certain major champion goes into IPO soon.
Re: Reasoning models reason well, until they don't
#26Earlier quoted context omitted.
I guess we mean here ”usefull reasoning” instead of the idiot-savant. I mean it’s a fair ask since these are marketed as _tools_ you can use to implement _industrial processes_ and even replace you human workers. In that I guess the model does not need to be the most reasonable intepreter of vague and poorly formulated user inputs but I think to improve a bit at least, to become usefull general appliances and not jus…
Thanks. So, people are extending "reasoning" to include making good decisions, rather than just solving logic problems. That makes sense to me that if people use that definition, LLMs are pretty bad at "reasoning". Although, I would argue that this is not reasoning at all, but rather "common sense" or the ability to have a broader perspective or think of the future. These are tasks that come with experience. That is…
Ah yes then, let me then unchain my LLM on those nasty unsolved math and logic problems I've absolutely not be struggling with in the course of my career.
Re: Reasoning models reason well, until they don't
#27> [...] recent studies show that transformers and LLMs fail catastrophically once reasoning problems exceed modest complexity. We revisit these findings through the lens of large reasoning models (LRMs) -- LLMs fine-tuned with incentives for step-by-step argumentation and self-verification This was the obvious outcome of the study (don't get me wrong, obvious outcomes are still worth having research on). "LRMs" *are*…
> The truly amazing thing is that reasoning models show ANY improvement at all compared to non-reasoning models, when they're the same exact thing.
Why is that amazing? It seems expected. Use a tool differently, get different results.
Re: Reasoning models reason well, until they don't
#28I wonder if we can get models to reason in a structured and verifiable way, like we have formal logic in math.
For that, you already have classical programming. It is great at formal logic math.
I think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that.
I actually believe it is technically possible, but is going to be very hard.
Re: Reasoning models reason well, until they don't
#29But I also fail catastrophically once a reasoning problem exceeds modest complexity.
Yes, but you are not a computer. There is no point building another human. We have plenty of them.
Re: Reasoning models reason well, until they don't
#30Earlier quoted context omitted.
Thanks. So, people are extending "reasoning" to include making good decisions, rather than just solving logic problems. That makes sense to me that if people use that definition, LLMs are pretty bad at "reasoning". Although, I would argue that this is not reasoning at all, but rather "common sense" or the ability to have a broader perspective or think of the future. These are tasks that come with experience. That is…
Ah yes then, let me then unchain my LLM on those nasty unsolved math and logic problems I've absolutely not be struggling with in the course of my career.
A lot of maths students would also struggle to contribute to frontier math problems, but we would still say they are reasoning. Their skill at reasoning might not be as good as professional mathematicians, but that does not stop us from recognising that they can solve logic problems without memorisation, which is a form of reasoning.
I am just saying that LLMs have demonstrated they can reason, at least a little bit. Whereas it seems other people are saying that LLM reasoning is flawed, which does not negate the fact that they can reason, at least some of the time.
Maybe generalisation is one area where LLM's reasoning is weakest though. They can be near-elite performance at nicely boxed up competition math problems, but their performance dramatically drops on real-world problems where things aren't so neat. We see similar problems in programming as well. I'd argue the progress on this has been promising, but other people would probably vehemently disagree with that. Time will tell.