Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FrontierMath was funded by OpenAI

lesswrong.com

Post reply on HN
Print view View original

21–30 of 212 posts

Re: FrontierMath was funded by OpenAI

#21

Post by aithrowawaycomm »

th1243127 wrote: post #9
jsheard wrote: post #4

Why do people keep taking OpenAIs marketing spin at face value? This keeps happening, like when they neglected to mention that their most impressive Sora demo involved extensive manual editing/cleanup work because the studio couldn't get Sora to generate what they wanted. https://news.ycombinator.com/item?id=40359425

It might be because (very few!) mathematicians like Terence Tao make positive remarks. I think these mathematicians should be very careful to use reproducible and controlled setups that by their nature cannot take place on GPUs in the Azure cloud. I have nothing against scientists promoting the Coq Proof Assistant. But that's open source, can be run at home and is fully reproducible.

Keep in mind those mathematicians were kept in the dark about the funding: it is incredibly unethical to invite a coauthor to your paper and not tell where the money came from.

It's just incredibly scummy behavior: I imagine some of those mathematicians would have declined the collaboration if the funding were transparent. More so than data contamination, this makes me deeply mistrustful of Epoch AI.

Re: FrontierMath was funded by OpenAI

#22

Post by refulgentis »

aithrowawaycomm wrote: post #21
th1243127 wrote: post #9

Earlier quoted context omitted.

It might be because (very few!) mathematicians like Terence Tao make positive remarks. I think these mathematicians should be very careful to use reproducible and controlled setups that by their nature cannot take place on GPUs in the Azure cloud. I have nothing against scientists promoting the Coq Proof Assistant. But that's open source, can be run at home and is fully reproducible.

Keep in mind those mathematicians were kept in the dark about the funding: it is incredibly unethical to invite a coauthor to your paper and not tell where the money came from. It's just incredibly scummy behavior: I imagine some of those mathematicians would have declined the collaboration if the funding were transparent. More so than data contamination, this makes me deeply mistrustful of Epoch AI.

I can't parse any of this, can you explain to a noob? I get lost immediately: funding, coauthor, etc. Only interpretation I've come to is I've missed a scandal involving payola, Terence Tao, and keeping coauthors off papers

Re: FrontierMath was funded by OpenAI

#24

Post by teaearlgraycold »

aithrowawaycomm wrote: post #15
lolinder wrote: post #10

A co-founder of Epoch left a note in the comments: > We acknowledge that OpenAI does have access to a large fraction of FrontierMath problems and solutions, with the exception of a unseen-by-OpenAI hold-out set that enables us to independently verify model capabilities. However, we have a verbal agreement that these materials will not be used in model training. Ouch. A verbal agreement. As the saying goes, those aren…

What's even more suspicious is that these tweets from Elliot Glazer indicate that they are still "developing" the hold-out set, even though elsewhere Epoch AI strongly implied this already existed: https://xcancel.com/ElliotGlazer/status/1880809468616950187 It seems to me that o3's 25% benchmark score is 100% data contamination.

This was my assumption all along.

Re: FrontierMath was funded by OpenAI

#25

Post by refulgentis »

jsheard wrote: post #4

Why do people keep taking OpenAIs marketing spin at face value? This keeps happening, like when they neglected to mention that their most impressive Sora demo involved extensive manual editing/cleanup work because the studio couldn't get Sora to generate what they wanted. https://news.ycombinator.com/item?id=40359425

Because the models have continually matched the quality they claim.

Ex. look how much work "very few" has to do in the sibling comment. It's like saying "very few physicists [Einstein/Feynman/Witten]"

Its conveniently impossible to falsify the implication that the inverse of "very few" say not positive things. i.e. that the vast majority say negative things

You have to go through an incredible level of mental gymnastics, involving many months of gated decisions, where the route chosen involved "gee, I know this is suspectable to confirmation bias, but...", to end up wondering why people think the models are real if OpenAI has access to data that includes some set of questions.

Re: FrontierMath was funded by OpenAI

#26

Post by energy123 »

lolinder wrote: post #17
WasimBhai wrote: post #16

I have been taking a course in AI policy and the O1 and the FrontierMath dataset has been an important mark for me to emphasize the world we are moving toward. It is incredibly sad to know about the conflict of interest here. However, those more knowledgeable, can you explain in plain words, does this revelation compromise OAI's claims regarding o3's performance on FrontierMath problems?

They have an oral agreement that OpenAI won't use the benchmark in training. Which means first and foremost you have to consider the possibility that they broke that oral agreement and actually included the problems in the training set. Even if they didn't, the fact that they had the problems means they could have selectively chosen the training set data to specialize in solving that class of problem, while still tec…

If OpenAI wanted the questions/solutions, there is going to be a reason for that. This data is not sitting in an unopened folder on Sam's computer.

There are a lot of ways you can use data to improve a model without directly training on it. A train/test validation loop, for example. Or as a wellspring for synthetic data generation. But all of these ways involve some level of data contamination, it's unavoidable.

Re: FrontierMath was funded by OpenAI

#27

Post by Imnimo »

My guess is that OpenAI didn't cheat as blatantly as just training on the test set. If they had, surely they could have gotten themselves an even higher mark than 25%. But I do buy the comment that they soft-cheated by using elements of the dataset for validation (which is absolutely still a form of data leakage). Even so, I suspect their reported number is roughly legit, because they report numbers on many benchmarks, and they have a good track record of those numbers holding up to private test sets.

What's much more concerning to me than the integrity of the benchmark number is the general pattern of behavior here from OpenAI and Epoch. We shouldn't accept secretly (even secret to the people doing the creation!) funding the creation of a benchmark. I also don't see how we can trust in the integrity of EpochAI going forward. This is basically their only meaningful output, and this is how they handled it?

Re: FrontierMath was funded by OpenAI

#28

Post by g-b-r »

numba888 wrote: post #7

if they used it in training it should be 100% hit. most likely they used it to verify and tune parameters.

Had they let it hit 100% it would have been obvious they had the data.

They've sure been careful to avoid that, by only using a portion of it or some other technique

Re: FrontierMath was funded by OpenAI

#29

Post by rrr_oh_man »

numba888 wrote: post #7

if they used it in training it should be 100% hit. most likely they used it to verify and tune parameters.

> if they used it in training it should be 100% hit.

Not necessarily, no.

A statistical model will attempt to minimise overall loss, generally speaking.

If it gets 100% accuracy on the training data it's usually an overfit. (Hugging the data points too tightly, thereby failing to predict real life cases)

Re: FrontierMath was funded by OpenAI

#30

Post by Vecr »

refulgentis wrote: post #22
aithrowawaycomm wrote: post #21

Earlier quoted context omitted.

Keep in mind those mathematicians were kept in the dark about the funding: it is incredibly unethical to invite a coauthor to your paper and not tell where the money came from. It's just incredibly scummy behavior: I imagine some of those mathematicians would have declined the collaboration if the funding were transparent. More so than data contamination, this makes me deeply mistrustful of Epoch AI.

I can't parse any of this, can you explain to a noob? I get lost immediately: funding, coauthor, etc. Only interpretation I've come to is I've missed a scandal involving payola, Terence Tao, and keeping coauthors off papers

Very few people were told the nature of the funding.
Post reply on HN