Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reasoning models reason well, until they don't

arxiv.org

Post reply on HN
Print view View original

131–140 of 224 posts

Re: Reasoning models reason well, until they don't

#131

Post by anal_reactor »

acdha wrote: post #124
anal_reactor wrote: post #39

I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.

The problem is consistency: AI tools usually produce output which _sounds_ like the top 10% but you have to read it carefully to find the bottom 10% parts. We’re not used to that because human performance isn’t that inconsistent and we use history and social factors: someone’s performance goes down when they’re really drunk, but they rarely show up to work in that state and it’s obvious enough that other people recog…

> We’re not used to that because human performance isn’t that inconsistent

It is. It's very common for socially apt people to bullshit through things they don't know, or outright want to hide.

Re: Reasoning models reason well, until they don't

#132

Post by acdha »

anal_reactor wrote: post #131
acdha wrote: post #124

Earlier quoted context omitted.

The problem is consistency: AI tools usually produce output which _sounds_ like the top 10% but you have to read it carefully to find the bottom 10% parts. We’re not used to that because human performance isn’t that inconsistent and we use history and social factors: someone’s performance goes down when they’re really drunk, but they rarely show up to work in that state and it’s obvious enough that other people recog…

> We’re not used to that because human performance isn’t that inconsistent It is. It's very common for socially apt people to bullshit through things they don't know, or outright want to hide.

That’s not inconsistent: your bluffer knows they’re making something up and is using their model of you to construct something they think you’ll believe. Someone who can do that isn’t going to suddenly forget how to count the number of letters in a word.

Re: Reasoning models reason well, until they don't

#133

Post by pessimizer »

antonvs wrote: post #20
monkeydust wrote: post #8

Earlier quoted context omitted.

But you recognise you are likely to fail and thus dont respond or redirect the problem to someone who has a greater likelihood of not failing.

I’ve had models “redirect the problem to someone who has a greater likelihood of not failing”. Gemini in particular will do this when it runs into trouble. I don’t find all these claims that models are somehow worse than humans in such areas convincing. Yes, they’re worse in some respects. But when you’re talking about things related to failures and accuracy, they’re mostly superhuman. For example, how many humans ca…

> I’ve had models “redirect the problem to someone who has a greater likelihood of not failing”. Gemini in particular will do this when it runs into trouble.

I have too, and I sense that this is something that has been engineered in rather than coming up naturally. I like it very much and they should do it a lot more often. They're allergic to "I can't figure this out" but hearing "I can't figure this out" gives me the alert to help it over the hump.

> But when you’re talking about things related to failures and accuracy, they’re mostly superhuman.

Only if you consider speed to failure and inaccuracy. They're very much subhuman in output, but you can make them retry a lot in a short time, and refine what you're asking them each time to avoid the mistakes they're repeatedly making. But that's you doing the work.

Re: Reasoning models reason well, until they don't

#134

Post by acuozzo »

riskable wrote: post #116

My hypothesis: This is why AI is fantastic as a coding assistant but not so great at other things. A software developer—after watching an AI model fail over and over again, trying to say, fix a difficult bug—will stop and approach the issue from a different angle. They'll take a closer look at what's going on, fiddle things around by hand, and that's usually enough to get over that hump of complexity (that the AI mod…

> AI is fantastic as a coding assistant

The extent to which this is true is a rough measure of how derivative your work is, no?

Re: Reasoning models reason well, until they don't

#136

Post by anal_reactor »

acdha wrote: post #132
anal_reactor wrote: post #131

Earlier quoted context omitted.

> We’re not used to that because human performance isn’t that inconsistent It is. It's very common for socially apt people to bullshit through things they don't know, or outright want to hide.

That’s not inconsistent: your bluffer knows they’re making something up and is using their model of you to construct something they think you’ll believe. Someone who can do that isn’t going to suddenly forget how to count the number of letters in a word.

You're wrong. Counting the number of letters in a word is a significantly more difficult task than lying, both for humans and LLMs. Imagine going to a ghetto and asking people "have you ever lied to someone and had them believe the lie", and ask them to spell "continuously". Children learn to lie before they learn to spell.

Re: Reasoning models reason well, until they don't

#138

Post by darkwater »

squidbeak wrote: post #135
darkwater wrote: post #129

Earlier quoted context omitted.

So, was Newtonian physics exact already?

> Science is exact without being finished

Being exact doesn't mean it is not an approximation, which was the initial topic. Being exact in science means that 2+2=4 and that can be demonstrated following a logical chain. But that doesn't make our knowledge of the universe exact. It is still an approximation. What it can be "exact" is how we obtain and reproduce the current knowledge we have of it.

Re: Reasoning models reason well, until they don't

#139

Post by ralfd »

conception wrote: post #100
krapht wrote: post #97

Earlier quoted context omitted.

Citation needed.

Searching for koko ape fraud seems to produce a lot.

> In his lecture, Sapolsky alleges that Patterson spontaneously corrects Koko’s signs: “She would ask, ‘Koko, what do you call this thing?’ and [Koko] would come up with a completely wrong sign, and Patterson would say, ‘Oh, stop kidding around!’ And then Patterson would show her the next one, and Koko would get it wrong, and Patterson would say, ‘Oh, you funny gorilla.’ ”

More weirdly was this lawsuit against Patterson:

> The lawsuit alleged that in response to signing from Koko, Patterson pressured Keller and Alperin (two of the female staff) to flash the ape. "Oh, yes, Koko, Nancy has nipples. Nancy can show you her nipples," Patterson reportedly said on one occasion. And on another: "Koko, you see my nipples all the time. You are probably bored with my nipples. You need to see new nipples. I will turn my back so Kendra can show you her nipples."[47] Shortly thereafter, a third woman filed suit, alleging that upon being first introduced to Koko, Patterson told her that Koko was communicating that she wanted to see the woman's nipples

There was a bonobo named Kanzi who learned hundreds of lexigrams. The main criticism here seems to be that while Kanzi truly did know the symbol for “Strawberry” he “used the symbol for “strawberry” as the name for the object, as a request to go where the strawberries are, as a request to eat some strawberries”. So no object-verb sentences and so no grammar which means no true language according to linguists.

https://linguisticdiscovery.com/posts/kanzi/

Re: Reasoning models reason well, until they don't

#140

Post by sodality2 »

MangoToupe wrote: post #127
pistoriusp wrote: post #80

Earlier quoted context omitted.

I saw a meme that I think about fairly often: Great apes have learnt sign language, and communicated with humans, since the 1960's. In all that time they've never asked human questions. They've never tried to learn anything new! The theory is that they don't know that there are entities that know things they don't. I like to think that AI are the great apes of the digital world.

> The theory is that they don't know that there are entities that know things they don't. This seems like a rather awkward way of putting it. They may just lack conceptualization or abstraction, making the above statement meaningless.

The exact title of the capacity is 'theory of mind' - for example, chimpanzees have a limited capacity for it in that they can understand others' intentions, but they seemingly do not understand false beliefs (this is what GP mentioned).

https://doi.org/10.1016/j.tics.2008.02.010

Post reply on HN