I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
[deleted]
Reasoning models reason well, until they don't
61–70 of 224 posts
Re: Reasoning models reason well, until they don't
#62Earlier quoted context omitted.
Don't they have a significant RL component? The "we'll just make it bigger" idea that was peddled a lot after GPT3.5 was nonsense, but that's not the only thing they're doing right now.
"We'll just make it bigger" works. RLVR just gives better performance gains and spends less inference compute - as long as you have a solid way of verifying the tasks. A simplified way of thinking about it is: pretraining gives LLMs useful features, SFT arranges them into useful configurations, RLVR glues them together and makes them work together well, especially in long reasoning traces. Makes sense to combine it a…
I'm not against "we'll make it bigger" (although it's as of yet unknown if it hits diminishing returns, 4.5 isn't exactly remembered as a great release), I'm against "we'll just (i.e. 'only') make it bigger".
I'm doubtful you'd have useful LLMs today if labs hadn't scaled in post-training.
Re: Reasoning models reason well, until they don't
#63Has any one ever found an ML/AI paper that make claims that RLMs can reason? When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.
Semantics schemantics.
Re: Reasoning models reason well, until they don't
#64LLMs falter because likelihood-driven pattern completion doesn’t enforce coherence across uncertainty (probability), representation (geometry), composition (category), and search (reasoning). To get robust reasoning, we need these layers to be explicit, typed, and mutually constraining—with verification and calibrated belief updates in the loop. I was interviewed about this recently, and mentioned the great work of a…
Richard Sutton's interview on Dwarkesh's podcast hit at this same point. The implicit world models in LLMs are insufficient.
Sutton still hasn't learned his own Bitter Lesson? ;)
Re: Reasoning models reason well, until they don't
#65I think the explanation is pretty simple, as I said in my earlier comment: https://news.ycombinator.com/item?id=44904107 I also believe the problem is we don't know what we want: https://news.ycombinator.com/item?id=45509015 If we could make LLMs to apply a modest set of logic rules consistently, it would be a win.
That's a pretty big "if". LLMs are by design entirely unlike GoFAI reasoning engines. It's also very debatable whether it makes any sense to try and hack LLMs into reasoning engines when you could just... use a reasoning engine. Or have the LLM to defer to one, which would play to their strength as translators.
Re: Reasoning models reason well, until they don't
#66Earlier quoted context omitted.
Do you? Don't you just halt and say this is too complex?
Some would consider that to be failing catastrophically. The task is certainly failed.
I would consider that detecting your own limits when trying to solve a problem is preferable to having the illusion of thinking that your solution is working and correct.
Re: Reasoning models reason well, until they don't
#67I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…
I think a good test of this seems to be to provide an image and get the model to predict what will happen next/if x occurs. They fail spectacularly at Rube-Goldberg machines. I think developing some sort of dedicated prediction model would help massively in extrapolating data. The human subconscious is filled with all sorts of parabolic prediction, gravity, momentum and various other fast-thinking paths that embed th…
Any example of that? One would think that predicting what comes next from an image is basically video generation, which works not perfect, but works somehow (Veo/Sora/Grok)
Re: Reasoning models reason well, until they don't
#68Has any one ever found an ML/AI paper that make claims that RLMs can reason? When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.
That would require the ability to understand what happens inside the system during inference when the output is created and they can't do that today.
There's no evidence to be had when we only know the inputs and outputs of a black box.
Re: Reasoning models reason well, until they don't
#69Earlier quoted context omitted.
For that, you already have classical programming. It is great at formal logic math.
I think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that. I actually believe it is technically possible, but is going to be very hard.
This is where you get the natural language tool to write the formal logic.
ChatGPT knows WebPPL really well for example.