More questions about whether researchers can trust OpenAI with unpublished math • Hacker News Board

Re: More questions about whether researchers can trust OpenAI with unpublished math

#781

Post by robotpepi » Fri, Sep 11, 2026, 8:52 AM UTC

jjwiseman wrote: post #423

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot…

> It's not obvious to me that's an unethical thing to do, if it happened as they described.

What!? Even if everything OpenAI said is accurate (big hypothesis there!), it's highly unethical to rush a solution because others have jsut had success. And that's the only beginning.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#782

Post by BrenBarn » Fri, Sep 11, 2026, 9:06 AM UTC

keeda wrote: post #716

SecretDreams wrote: post #700

Earlier quoted context omitted.

> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. No need to be mysterious. State what reasons you think these are in plain English?

Being displaced from a vocation that they either have dedicated their professional lives getting good at, or was their livelihood, or likely, both. I think all other complaints from all other people in all their myriad variations stem from this core reason. Even if people don't realize it themselves. Like, if these models had trained on the entirety of human knowledge and art, and then turned out to be absolutely use…

I'm not sure that's true. Like if they produced nothing but the worst of the slop they're currently producing, a lot of people would still be bothered by that just because of the sheer volume of such slop that can now be produced.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#783

Post by YeGoblynQueenne » Fri, Sep 11, 2026, 9:24 AM UTC

ImPostingOnHN wrote: post #429

YeGoblynQueenne wrote: post #393

Earlier quoted context omitted.

That's a jingle fallacy. *Jingle-jangle fallacies are erroneous assumptions that either two different things are the same because they bear the same name (jingle fallacy); or two identical or almost identical things are different because they are labeled differently (jangle fallacy).[1][2][3] The term was coined by Truman Lee Kelley in his 1927 book Interpretation of educational measurements.[4] In research, a jangle…

You are simply incorrect. It is not a fallacy of that type, or any other type, because the words do, in fact, mean the same thing, as multiple people have pointed out here. Whether referring to chatbots or people, "prompt" means "to move to action". If you have some reliable source supporting your unilateral claims that "prompt" does not mean this, please share. Otherwise, the consensus seems to be contrary to your c…

Why do I need a source? An LLM prompt does not "move to action", because an LLM does not act. People act, animals act, software doesn't act. Acting implies volition and volition implies cognition and if you think that LLMs have those things then you're the one who should provide a source for your claim.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#784

Post by YeGoblynQueenne » Fri, Sep 11, 2026, 9:43 AM UTC

boothby wrote: post #518

YeGoblynQueenne wrote: post #369

Earlier quoted context omitted.

>> If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great! It's perhaps great in the short term although it's not very clear who it's great for. I'm not sure mathematicians find it all so great, I mean. In the l…

I was pretty depressed when I read about what happened with Navier Stokes this morning. The Clay Math prizes were a significant motivation through my math career, and I know a lot of computer scientists and physicists that feel similarly. I didn't think I was gonna resolve P vs NP or the BSD conjecture, but I did really research that felt like I was working towards something incredible. What is the younger generation…

This is no time to despair. It's the time to take a stand. If you don't want to see your discipline go the way of the dodo, then do something about it.

I don't know what you should do because I'm not a mathematician. But superintelligence schmuperintelligence. We didn't stop running because we have cars or playing chess or Go because there's chess and Go engines. Even more so than chess there's no point in maths unless it's people doing it, for other people. AI maths makes no sense, like AI art makes no sense, because those are things that people enjoy and can do pretty damn well ourselves so there's no point to automate them away. We gotta stop that bullshit, and we can stop it. And if we don't, if we just sit around and wait for OpenAI and Anthropic to destroy society then that's not their fault but ours.

Sorry, I'm not great at pep talks. Those are brave men. Let's go kill them!

Re: More questions about whether researchers can trust OpenAI with unpublished math

#785

Post by enyone » Fri, Sep 11, 2026, 9:47 AM UTC

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

I think OP's analogy is bad. The difference of OpenAI when comparing to human collaborator is the possibility to replicate once learned skill. Imagine if any single human collaborator learns a skill it is immediately a skill of any human collaborator.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#786

Post by blensor » Fri, Sep 11, 2026, 10:10 AM UTC

ThalesX wrote: post #472

blensor wrote: post #48

Earlier quoted context omitted.

Let's turn this question around. If I have infinite money to progress whatever problem solution I want but I always wait until I have an unfair advantage to get credit for whatever problem was just at the brink of a breakthrough anyway by sniping the last steps. Am I actually doing a good thing or would it be better to let it run it's natural course and spend the money somewhere it's actually needed?

Sort of like Apple takes validated market products and snipes the last steps to an actual good UX (at least in theory)?

You are probably not too far off given that Apple sometimes releases their own version of a product that a developer established on their platform

The thing is, everyone knows that Apple does that and Apple doesn't care if people have a beef with them.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#787

Post by charlieyu1 » Fri, Sep 11, 2026, 10:14 AM UTC

snaking0776 wrote: post #713

user43928 wrote: post #614

Earlier quoted context omitted.

That's also incorrect. My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result. This is the paper where they did not want the Anthropic employee collaborating. Not their work.

I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.

Looks like a lot of insecurity from OpenAI. At the top level researchers move around at their own will, you don’t decide who they work for

Re: More questions about whether researchers can trust OpenAI with unpublished math

#788

Post by alper » Fri, Sep 11, 2026, 10:18 AM UTC

It's fine. They only need to steal the discoveries long enough to go IPO, then the companies will enshittify and the scientists can go back to doing their original work (which the models can't do anyway).

Re: More questions about whether researchers can trust OpenAI with unpublished math

#789

Post by alper » Fri, Sep 11, 2026, 10:20 AM UTC

jjwiseman wrote: post #423

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot…

> not claiming that the model wasn't trained on those sessions

The math group inside OpenAI may be training or fine tuning their own models which given some reward functions would definitely bias their usage of the training data towards things that look like math.

You can launder all of it without a human "directly" doing anything.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#790

Post by user43928 » Fri, Sep 11, 2026, 10:21 AM UTC

irthomasthomas wrote: post #440

famouswaffles wrote: post #371

Earlier quoted context omitted.

OpenAI have come out and said: >The Wednesday evening statement from OpenAI was more emphatic: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.” >The statement added, “After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way…

Is there a reason they scoped that so narrowly to Buckmaster/codex/2 months two people worked on this for a year before the breakthrough. Perhaps that earlier work reduced the search space sufficiently to brute force the problem with 10,000 agents?

The comment you replied to quoted "no user inputs after July 3rd" with no restriction to Buckmaster or Codex.

Obviously the result of OpenAI's investigation was that no usage data has interacted with the system after that date.

What else do you expect them to investigate?

If Buckmaster and co. provide their chats, OpenAI could potentially search for them in the anonymized opted-in usage data. Then they could say if any data has been used.

By all accounts individual usage data does not have the direct impact on the model most here fantasize about. To prove this, OpenAI would need to do new training runs to replicate the system used minus the particular usage data in question, if it exists, and then benchmark this on the problem again.

Potentially multiple times, in order to reach a conclusion.

The cost might be in the hundreds of millions.