Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

30% drop in O1-preview accuracy when Putnam problems are slightly variated

openreview.net

Post reply on HN
Print view View original

361–370 of 558 posts

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#361

Post by wongarsu »

roughly wrote: post #347
wongarsu wrote: post #343

Earlier quoted context omitted.

If GP's hypothesis was "it fails for small variations of the input, like this one", then testing that hypothesis with that exact variation on a couple models seems fair and scientific. Testing it with more variations until one fails feels a bit like p-hacking. You'd need to engage in actual statistics to get reliable results from that, beyond "If I really try, I can make it fail". Which would be a completely differen…

Except that if the model genuinely was reasoning about the problem, you could test it with every variation of materials and weights in the world and it would pass. Failing that problem at all in any way under any conditions is a failure of reasoning.

We are pretty certain that humans can reason, yet they are sometimes wrong. Even if you give them the same problem over and over again with slight variations.

LLMs get things wrong due to different factors than humans (humans lose focus, LLMs have randomness applied when sampling their responses to improve results). But clearly we have to choose a goal somewhat below 100% if we want a test that doesn't conclude that humans are incapable of reasoning.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#362

Post by gmueckl »

zaroth wrote: post #315
deeviant wrote: post #210

Earlier quoted context omitted.

I don't believe that is the model that you used. I wrote a script and pounded 01 mini and gpt 4 with a wide vareity of tempature and top_p parameters, and was unable to get it to give the wrong answer a single time. Just a whole bunch of: (openai-example-py3.12) :~/code/openAiAPI$ python3 featherOrSteel.py Response 1: A 10.01-pound bag of fluffy cotton is heavier than a 9.99-pound bag of steel ingots. Response 2: A 1…

Downvoted for… too conclusively proving OP wrong?

Down voted for not actually countering the argument in question? The script doesn't alter the phrasing of the question itself. It just generates a randomized, irrelevant preamble.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#363

Post by maratc »

jerf wrote: post #197

I remember when this stuff was all coming out and people were finally excited about ChatGPT getting the problem with "which is heavier, a 10 pound bag of feathers or a 10 pound bag of bricks?" problem correct. But of course it got it correct. It was in the training set. Vary the problem slightly by just changing the nouns, or changing the numbers so that one in fact was heavier than the other, and performance went al…

Still waiting for some reasoning about an upward buoyancy force on the bags, which is equal to the weight of the air displaced by the bags, and the force acting on the cotton is bigger simply because the bag of cotton displaces more air (because it "appears larger due to its fluffy nature".)

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#364

Post by mung_daal »

jerf wrote: post #197

I remember when this stuff was all coming out and people were finally excited about ChatGPT getting the problem with "which is heavier, a 10 pound bag of feathers or a 10 pound bag of bricks?" problem correct. But of course it got it correct. It was in the training set. Vary the problem slightly by just changing the nouns, or changing the numbers so that one in fact was heavier than the other, and performance went al…

[dead]

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#365

Post by chongli »

Horffupolde wrote: post #354
AnimalMuppet wrote: post #335

Earlier quoted context omitted.

Can you explain? An ounce of gold is heavier than an ounce of feathers, because the "ounce of gold" is a troy ounce, and the "ounce of feathers" is an avoirdupois ounce. But that shouldn't be true between feathers and bricks - they're both avoirdupois.

Feathers are less dense so they have higher buoyancy in air, reducing their weight.

Pounds are a unit of weight, not of mass. 10 lbs of feathers is whatever amount of feathers causes a scale to display 10 lbs. If the scale also displays 10 lbs for the quantity of bricks, then they weigh the same, regardless of any differences in mass.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#366

Post by Groxx »

ttul wrote: post #352
s1mplicissimus wrote: post #169

Earlier quoted context omitted.

> 1. OpenAI has confirmed it’s not in their train (unlike putnam where they have never made any such claims) Companies claim lots of things when it's in their best financial interest to spread that message. Unfortunately history has shown that in public communications, financial interest almost always trumps truth (pick whichever $gate you are aware of for convenience, i'll go with Dieselgate for a specific example).…

OpenAI’s credibility is central to its business: overstating capabilities risks public blowback, loss of trust, and regulatory scrutiny. As a result, it is unlikely that OpenAI would knowingly lie about its models. They have much stronger incentives to be as accurate as possible—maintaining their reputation and trust from users, researchers, and investors—than to overstate capabilities for a short-term gain that woul…

>OpenAI’s credibility is central to its business: overstating capabilities risks public blowback, loss of trust, and regulatory scrutiny.

Uh huh. Kinda like what's happening right now?

They're marketing blow-hards. Everyone knows it. They've been wildly over-stating capabilities (and future capabilities!) as long as Altman has had power, and arguably longer.

They'll do it as long as they can get away with it, because that's all that is needed to make money on it. Factual accuracy rarely impacts the market when it's so hype-driven, especially when there is still some unique utility in the product.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#367

Post by jiggawatts »

jerf wrote: post #197

I remember when this stuff was all coming out and people were finally excited about ChatGPT getting the problem with "which is heavier, a 10 pound bag of feathers or a 10 pound bag of bricks?" problem correct. But of course it got it correct. It was in the training set. Vary the problem slightly by just changing the nouns, or changing the numbers so that one in fact was heavier than the other, and performance went al…

A real acid test is to use numbers that confuse LLMs such as:

“Which is heavier: a bag with 9.9 pounds of fluff or a bag with 9.11 pounds of steel block?”

Try swapping the numbers and the materials, run each test at least 10x times.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#368

Post by whakim »

roughly wrote: post #347
wongarsu wrote: post #343

Earlier quoted context omitted.

If GP's hypothesis was "it fails for small variations of the input, like this one", then testing that hypothesis with that exact variation on a couple models seems fair and scientific. Testing it with more variations until one fails feels a bit like p-hacking. You'd need to engage in actual statistics to get reliable results from that, beyond "If I really try, I can make it fail". Which would be a completely differen…

Except that if the model genuinely was reasoning about the problem, you could test it with every variation of materials and weights in the world and it would pass. Failing that problem at all in any way under any conditions is a failure of reasoning.

I’m not really sure what you’re trying to say here - that LLMs don’t work like human brains? We don’t need to conduct any analyses to know that LLMs don’t “know” anything in the way humans “know” things because we know how LLMs work. That doesn’t mean that LLMs aren’t incredibly powerful; it may not even mean that they aren’t a route to AGI.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#369

Post by roughly »

wongarsu wrote: post #361
roughly wrote: post #347

Earlier quoted context omitted.

Except that if the model genuinely was reasoning about the problem, you could test it with every variation of materials and weights in the world and it would pass. Failing that problem at all in any way under any conditions is a failure of reasoning.

We are pretty certain that humans can reason, yet they are sometimes wrong. Even if you give them the same problem over and over again with slight variations. LLMs get things wrong due to different factors than humans (humans lose focus, LLMs have randomness applied when sampling their responses to improve results). But clearly we have to choose a goal somewhat below 100% if we want a test that doesn't conclude that…

The difference is we _know_ that LLMs are fancy stochastic models, we don't know that they're capable of reasoning, and the null hypothesis is that they're not (because we know what they _are_ - we built them) - any "reasoning" is an emergent property of the system, not something we built them to do. In that case, evidence they're not reasoning - evidence they're stochastic parrots doing a performance of reasoning - weighs heavier, because the performance of reasoning fits into what we know they can do, whereas genuine reasoning would be something new to the model.

There's deeper philosophical questions about what reasoning actually _is_, and LLMs have made those sharper, because they've shown it's clearly possible for a complex statistical model to generate words that look like reasoning, but the question is whether there's a difference between what they're doing and what humans are doing, and evidence that they're _not_ reasoning - evidence that they're just generating words in specific orders - weighs heavily against them.

Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated

#370

Post by bee_rider »

nwienert wrote: post #349
wongarsu wrote: post #343

Earlier quoted context omitted.

If GP's hypothesis was "it fails for small variations of the input, like this one", then testing that hypothesis with that exact variation on a couple models seems fair and scientific. Testing it with more variations until one fails feels a bit like p-hacking. You'd need to engage in actual statistics to get reliable results from that, beyond "If I really try, I can make it fail". Which would be a completely differen…

I feel like I'm almost 100% certain that the smart guys at OpenAI have added many more variations of the problem to their training set since OP did his failing test, so it doesn't surprise me at all to know that this exact one now passes. In fact, in my use of o1 it's incredibly clear that it still has the same problems. It's incredibly common that the second I ask for someone even slightly outside the training set,…

Can’t really do science with closed source software, right? Who knows what’s in there.
Post reply on HN