Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reasoning models reason well, until they don't

arxiv.org

Post reply on HN
Print view View original

91–100 of 224 posts

Re: Reasoning models reason well, until they don't

#91

Post by pimeys »

My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

I just got this from codex yesterday:

"I wasn’t able to finish; no changes were shipped."

And it's not the first time.

Re: Reasoning models reason well, until they don't

#92

Post by BOOSTERHIDROGEN »

pistoriusp wrote: post #80
My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

I saw a meme that I think about fairly often: Great apes have learnt sign language, and communicated with humans, since the 1960's. In all that time they've never asked human questions. They've never tried to learn anything new! The theory is that they don't know that there are entities that know things they don't. I like to think that AI are the great apes of the digital world.

Does that means intelligent is soul? Then we will never achieve AGI.

Re: Reasoning models reason well, until they don't

#93

Post by moritzwarhier »

From the abstract:

> some even claiming they are capable of generalized reasoning and innovation in reasoning-intensive fields such as mathematics, physics, medicine, and law. However, by more carefully scaling the complexity of reasoning problems, we show existing benchmarks actually have limited complexity

Can someone ELI5 what the definitions of reasoning and complexity are here?

I see they seem to focus on graph problems and representing problems as graph problems. But I didn't completely read the paper or understand it in depth. I skimmed some parts that seem to address this question (e.g. section 5 and the Introduction), but maybe there are simpler definitions that elude me.

Surely they don't mean "computational complexity"?

And what exactly is "reasoning"?

I'm aware of philosophical logic and strict logic that can be applied to natural language arguments.

But have we already agreed on a universal scale that grades answers to questions about the physical world? Or is this about mathematical reasoning?

Mixing all of this together always irks me when it comes to these AI "benchmarks". But apparently people see value in these?

I know my question isn't new.

To me it seems, that when we leave the mathematical realms, it quickly becomes fuzzy what correct "reasoning" should be.

People can be convincing and avoid obious logical fallacies, and still make wrong conclusions... or conclusions that run counter to assumed goals.

Re: Reasoning models reason well, until they don't

#94

Post by code_martial »

tomlockwood wrote: post #87
alyxya wrote: post #23

The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from som…

So the answer is a few more trillion?

It’s a worthwhile answer if it can be proven correct because it means that we’ve found a way to create intelligence, even if that way is not very efficient. It’s still one step better than not knowing how to do so.

Re: Reasoning models reason well, until they don't

#95

Post by amelius »

My_Name wrote: post #45

I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…

The problem is that the training data doesn't contain a lot of "I don't know".

Re: Reasoning models reason well, until they don't

#96

Post by tempfile »

hirako2000 wrote: post #51

Has any one ever found an ML/AI paper that make claims that RLMs can reason? When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.

I don't understand what point you are making. Doesn't the name "Reasoning language models" claim that they can reason? Why do you want to see it explicitly written down in a paper?

Re: Reasoning models reason well, until they don't

#97

Post by krapht »

20k wrote: post #90
pistoriusp wrote: post #80

Earlier quoted context omitted.

I saw a meme that I think about fairly often: Great apes have learnt sign language, and communicated with humans, since the 1960's. In all that time they've never asked human questions. They've never tried to learn anything new! The theory is that they don't know that there are entities that know things they don't. I like to think that AI are the great apes of the digital world.

Its worth noting that the idea that great apes have learnt sign language is largely a fabrication by a single person, and nobody has ever been able to replicate this. All the communication has to be interpreted through that individual, and anyone else (including people that speak sign language) have confirmed that they're just making random hand motions in exchange for food They don't have the dexterity to really sig…

Citation needed.

Re: Reasoning models reason well, until they don't

#98

Post by iq176 »

squidbeak wrote: post #83
flavaflav2 wrote: post #76

Earlier quoted context omitted.

Life and a lot in our universe is non-deterministic. Some people assume science and mathematics are some universal truths rather than imperfect agreed upon understandings. Similarly many assume humans can be controlled through laws, penalties, prisons, propaganda, coercion, etc. But terrible things happen. Yes, if you set up the gutter-rails in your bowling lane, you can control the bowling ball unless it is thrown o…

No, Science is a means of searching for those truths - definitely not some 'agreed upon understanding'. It's backed up by experimentation and reproducible proofs. You also make a huge bogus leap from science to humanities.

Scientific method is the process. Science itself includes the study and compendium of understandings, based on a belief system that includes shared understandings just like mathematics. The foundation of these are philosophical beliefs that we can know and understand these things. For example, on a metaphysical level, if the world around us were a simulation, then science could provide understandings about that simulated universe, but not about that which is simulating it.

Re: Reasoning models reason well, until they don't

#99

Post by joncrocks »

krapht wrote: post #97
20k wrote: post #90

Earlier quoted context omitted.

Its worth noting that the idea that great apes have learnt sign language is largely a fabrication by a single person, and nobody has ever been able to replicate this. All the communication has to be interpreted through that individual, and anyone else (including people that speak sign language) have confirmed that they're just making random hand motions in exchange for food They don't have the dexterity to really sig…

Citation needed.

https://en.wikipedia.org/wiki/Great_ape_language#Criticism_a... - Not word for word, but certainly casting doubt that apes were ever really communicating in the way that people may have thought.

Re: Reasoning models reason well, until they don't

#100

Post by conception »

krapht wrote: post #97
20k wrote: post #90

Earlier quoted context omitted.

Its worth noting that the idea that great apes have learnt sign language is largely a fabrication by a single person, and nobody has ever been able to replicate this. All the communication has to be interpreted through that individual, and anyone else (including people that speak sign language) have confirmed that they're just making random hand motions in exchange for food They don't have the dexterity to really sig…

Citation needed.

Searching for koko ape fraud seems to produce a lot.
Post reply on HN