Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reasoning models reason well, until they don't

arxiv.org

Post reply on HN
Print view View original

151–160 of 224 posts

Re: Reasoning models reason well, until they don't

#151

Post by gpderetta »

dymk wrote: post #148
usrbinbash wrote: post #107

Earlier quoted context omitted.

> They are good at repeating their training data, not thinking about it. Which shouldn't come as a surprise, considering that this is, at the core of things, what language models do: Generate sequences that are statistically likely according to their training data.

This is too large of an oversimplification of how an LLM works. I hope the meme that they are just next token predictors dies out soon, before it becomes a permanent fixture of incorrect but often stated “common sense”. They’re not Markov chains.

Indeed, they are next token predictors, but this is a vacuous statement because the predictor can be arbitrary complex.

Re: Reasoning models reason well, until they don't

#152

Post by usrbinbash »

code_martial wrote: post #94
tomlockwood wrote: post #87

Earlier quoted context omitted.

So the answer is a few more trillion?

It’s a worthwhile answer if it can be proven correct because it means that we’ve found a way to create intelligence, even if that way is not very efficient. It’s still one step better than not knowing how to do so.

> if it can be proven correct

Then the first step would be to prove that this works WITHOUT needing to burn through the trillions to do so.

Re: Reasoning models reason well, until they don't

#153

Post by cryptonym »

vidarh wrote: post #71
hansmayer wrote: post #26

Earlier quoted context omitted.

Ah yes then, let me then unchain my LLM on those nasty unsolved math and logic problems I've absolutely not be struggling with in the course of my career.

I've "unchained" my LLM on a lot of problems that I probably could solve, but that would take me time I don't have, and that it has solved in many case faster than I could. It may not be good enough to solve problems that are beyond us for most of us, but it certainly can solve a lot of problems for a lot of us that have gone unsolved for lack of resources.

Can solve problems you already know how to solve, if you micro-manage it and it'll BS a lot on the way.

If this is the maximum AGI-PhD-LRM can do, that'll be disappointing compared to investments. Curious to see what all this will become in few years.

Re: Reasoning models reason well, until they don't

#154

Post by cryptonym »

vidarh wrote: post #72
sothatsit wrote: post #30

Earlier quoted context omitted.

A lot of maths students would also struggle to contribute to frontier math problems, but we would still say they are reasoning. Their skill at reasoning might not be as good as professional mathematicians, but that does not stop us from recognising that they can solve logic problems without memorisation, which is a form of reasoning. I am just saying that LLMs have demonstrated they can reason, at least a little bit.…

Thank you for picking at this. A lot of people appear to be - often not consciously or intentionally - setting the bar for "reasoning" at a level many or most people would not meet. Sometimes that is just a reaction to wanting an LLM that is producing result that is good for their own level. Sometimes it reveals a view of fellow humans that would be quite elitist if stated outright. Sometimes it's a kneejerk attempt…

Computers have forever been doing stuff people can't do.

The real question is how useful this tool is and if this is as transformative as investors expect. Understanding its limits is crucial.

Re: Reasoning models reason well, until they don't

#155

Post by brap »

nl wrote: post #69
brap wrote: post #28

Earlier quoted context omitted.

I think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that. I actually believe it is technically possible, but is going to be very hard.

This is where you get the natural language tool to write the formal logic. ChatGPT knows WebPPL really well for example.

You will need a formal language first.

Take this statement for example:

>ChatGPT knows WebPPL really well

What formal language can express this statement? What will the text be parsed into? Which transformations can you use to produce other truthful (and interesting) statements from it? Is this flexible enough to capture everything that can be expressed in English?

The closest that comes to mind is Prolog, but it doesn’t really come close.

Re: Reasoning models reason well, until they don't

#157

Post by egberts1 »

bob_theslob646 wrote: post #81
egberts1 wrote: post #73

It's simple. Don't ingest more than 40KB at a time into its LLM's RAG pipe and its hallucination goes way, way down. Preferably like not at the start and best not to do more than 40KB at a time at all. That's how I learned how to deal with nftables' 120KB parser_bison.y file by breaking them up into clean sections. All of a sudden, a fully-deterministic LL(1) full semantic pathway of nftables' CLI syntax appears befo…

Why 40kb?

Cheap public offering of their expensive data center is that sweet spot and cutoff at 40KB.

Re: Reasoning models reason well, until they don't

#158

Post by anal_reactor »

acdha wrote: post #145
anal_reactor wrote: post #136

Earlier quoted context omitted.

You're wrong. Counting the number of letters in a word is a significantly more difficult task than lying, both for humans and LLMs. Imagine going to a ghetto and asking people "have you ever lied to someone and had them believe the lie", and ask them to spell "continuously". Children learn to lie before they learn to spell.

> Counting the number of letters in a word is a significantly more difficult task than lying No, it’s not - you don’t even need to be literate to count symbols - but also consider the complexity of the second task and how many skills each requires: unlike counting letters, lying isn’t simple confabulation and requires a theory of mind and some kind of goal. A child who lies to avoid trouble is doing that because they…

Sure, let's stick to counting symbols. When I need to count something, there's a decent chance I'll get lost if I count beyond 10, and beyond 20 I'll get lost for sure. Even below 10, when I count it's one-two-three-four-five-six-seven-eight-nine items. But when I lie I do it instantaneously, without altering the pace of the conversation. I can come up with a believable lie within the brief period between someone saying something to me, and the moment I'm expected to respond. No way I'd be able to count 10 items that fast.

Pirahã language doesn't even have numerals - that's an extreme case, but there quite a few languages where people stop counting beyond certain small number and just say "a lot". Same people though don't have issues lying to one another. Let that sink in for a while - fully grown-ass adults, fully capable of functioning in their society, not capable of counting one-two-three because the concept is beyond them.

What I'm trying to say is that all of those "requires theory of mind" statements are probably true but completely irrelevant because humans (and LLMs) have "hardware acceleration" of whatever it takes to lie, meanwhile counting is an abstract idea that requires to use the brain in a way it didn't evolve to be used. Similarly, LLMs cannot count if they aren't connected to a math engine - not because they're stupid, but because counting is really difficult.

Re: Reasoning models reason well, until they don't

#159

Post by vidarh »

cryptonym wrote: post #153
vidarh wrote: post #71

Earlier quoted context omitted.

I've "unchained" my LLM on a lot of problems that I probably could solve, but that would take me time I don't have, and that it has solved in many case faster than I could. It may not be good enough to solve problems that are beyond us for most of us, but it certainly can solve a lot of problems for a lot of us that have gone unsolved for lack of resources.

Can solve problems you already know how to solve, if you micro-manage it and it'll BS a lot on the way. If this is the maximum AGI-PhD-LRM can do, that'll be disappointing compared to investments. Curious to see what all this will become in few years.

I'm not usually micro-managing it, that's the point.

I sometimes do on problems where I have particular insight, but I mostly find it is far more effective to give it test cases and give it instructions on how to approach a task, and then let it iterate with little to no oversight.

I'm letting Claude Code run for longer and longer with --dangerously-skip-permissions, to the point I'm pondering rigging up something to just keep feeding it "continue" and run it in parallel on multiple problems.

Because at least when you have a good way of measuring success, it works.

Re: Reasoning models reason well, until they don't

#160

Post by alyxya »

jeremyjh wrote: post #77
alyxya wrote: post #23

The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from som…

The burden of evidence here is on you. They don’t need to prove LRMs can’t scale to meet these problems; their only claim is current models can’t handle these problems. Others will take this up as a challenge - and chances may be good they will overcome it. This is how science works.

They can’t claim current models aren’t able to handle these problems if they didn’t use a setup similar to coding agents like Claude Code and OpenAI Codex. Using a suboptimal setup is akin to verbally telling a person the whole reasoning problem without letting them write down notes and expecting them to memorize and solve it after only hearing it once.
Post reply on HN