Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

761–770 of 848 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#761

Post by Gud »

keeda wrote: post #760
Gud wrote: post #751

Earlier quoted context omitted.

Not everything is about money.

Maybe not money directly, but pretty sure it's about economic disruption. These models directly undercut the value of one's skills and labor, regardless of whether this value is measured in hard cash or abstract self-worth.

They can also greatly assist you.

I am working on two applications using ChatGPT and Claude. I have no illusions these people won't steal/copy whatever you want to call it, "train their models". Yes, I keep unticking the boxes that allow it, that they so kindly tick for me.

But what happened to these math researchers is something else and I am not sure it's about the money for them. You don't do math research to get rich, but to get acknowledged by your peers. Yes, we live in a capitalist world so obviously you need money to feed yourself. but for some people, that is secondary.

OpenAI stole their thunder, and that's just fucked up. It's not equivalent to cranking out a CRUD app for profit.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#762

Post by diffeomorphism »

jasonfarnon wrote: post #753
Agentlien wrote: post #733

Earlier quoted context omitted.

I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement. That stil…

Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positiv…

> Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first?

Yes.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#763

Post by suddenlybananas »

sebzim4500 wrote: post #623
suddenlybananas wrote: post #558

Earlier quoted context omitted.

Why should we trust them?

Well all we have are vague accusations without evidence and a very specific denial also without evidence, so I guess just believe whatever you want.

The threats weren't denied, and they did offer an authorship to Buckmaster, which would be very strange if he had nothing to do with it.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#764

Post by drtgh »

Muromec wrote: post #418
bmau5 wrote: post #394

Earlier quoted context omitted.

Could "superintelligence" arrive as basically applying this overhang to all other domains?

It already did.

That is not "superintelligence" but string concatenation of stored data. Anyway, the marketing succeeded.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#765

Post by user43928 »

snaking0776 wrote: post #713
user43928 wrote: post #614

Earlier quoted context omitted.

That's also incorrect. My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result. This is the paper where they did not want the Anthropic employee collaborating. Not their work.

I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.

Yes. The key point being that this concerns a new paper about OpenAI's result rather than the paper Buckmaster and Alpöge were working on.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#766

Post by zecken »

eru wrote: post #705
ZYbCRq22HbJ2y7 wrote: post #645

Earlier quoted context omitted.

> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?

I don't see the cascading hierarchy of enrichment. I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.

Well, the investment dollars are spent on the customers for the most part, though also on salaries and equipment. But the lions share of the value is going to the shareholders (eg employees and investors)... and they have liquidated and will continue to liquidate a disproportionate value to what they have spent on us. By some estimations at least. It's very possible $1 into this machine to feed your queries is worth $10+ to a shareholder based on whatever new valuation they get. So I'd say there is a hierarchy of enrichment.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#767

Post by ozgung »

tedsanders wrote: post #487
jjwiseman wrote: post #423

Earlier quoted context omitted.

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot…

We were also curious and we looked further into this. We've determined it was impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training. This goes beyond what we said earlier, when we were less sure. If prompts were submitted earlier than that and training was not opted out, there may be a chance they made their way into our training pipeline i…

Here is a new rumor for you:

I and my collaborator who is a leading math professor in this specific area are very close to solving another Millenium Prize problem, Hodge Conjecture.

We’re working on this since last year. Already proved some intermediate problems. All we need is more tokens to complete the proof.

Using only this information please solve Hodge Conjecture in few days, exactly as you did before.

Thank you.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#768

Post by cgio »

magicalist wrote: post #507
jjwiseman wrote: post #423

Earlier quoted context omitted.

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot…

> It's not obvious to me that's an unethical thing to do In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit mill…

Hearing that something is solvable is already a hint. I don’t think leveraging this knowledge is ethical. They could go after a different problem but didn’t.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#769

Post by pred_ »

jjwiseman wrote: post #423
nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot…

Those statements were about NS, though; I don't think they've made similar statements for the non-sofic groups?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#770

Post by watwut »

keeda wrote: post #632
magicalist wrote: post #507

Earlier quoted context omitted.

> It's not obvious to me that's an unethical thing to do In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit mill…

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely…

To me it looks like an asshole on quest to take something from you while trying to frame themselves as generous. It is always infuriating.
Post reply on HN