Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

711–720 of 846 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#711

Post by keeda »

what wrote: post #682
keeda wrote: post #632

Earlier quoted context omitted.

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely…

> So they reached out to the other researchers as an attempt to share the credit This isn’t at all what happened? What are you talking about?

That was in response to this part of OP's post:

> If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.

From what I can tell, both OpenAI and the researchers agree on this meeting happening, except both sides clearly have very different interpretations of what happened and why.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#712

Post by grumple »

ModernMech wrote: post #430
kzz102 wrote: post #360

Earlier quoted context omitted.

On your second point: there is a more plausible explanation which David Bessis calls the "overhang". The short version is that there is a large amount of relatively low hanging fruits in mathematics, because no human has broad enough knowledge and enough time to try them all. AI is not constraint by that, and therefore can systematically pluck all those low hanging fruits. Quote: "The Overhang consists of the unreali…

> no human has broad enough knowledge and enough time to try them all. The other part is, humans don’t really want to fund other humans doing this. Very few want to be a math major; and of those that do, fewer complete a grad degree; and for those that do get grad degrees, there’s scant few research jobs; and for those who do get jobs there’s hardly any research funding to go around. There does seem to be unlimited m…

I’m assuming the reported 22 million dollars worth of tokens used to solve this particular problem is far more than what humans have paid to solve it previously. So I think you’re correct.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#713

Post by snaking0776 »

user43928 wrote: post #614
dwaltrip wrote: post #513

Earlier quoted context omitted.

I heard they also tried to strong-arm them into removing the name of their collaborator who happened to work at a different company (Anthropic)... I haven't looked into it myself, but if true, that seems incredibly scummy.

That's also incorrect. My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result. This is the paper where they did not want the Anthropic employee collaborating. Not their work.

I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#714

Post by timcobb »

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

> but a full data trail of all inputs is difficult to trace through.

Great use case for AI agents

Re: More questions about whether researchers can trust OpenAI with unpublished math

#715

Post by zarzavat »

keeda wrote: post #632
magicalist wrote: post #507

Earlier quoted context omitted.

> It's not obvious to me that's an unethical thing to do In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit mill…

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely…

Even if you judge OpenAI solely on their public communications it still sounds really bad.

That they heard a rumour that a major open problem had been solved, so they decided to try and scoop the other mathematicians while they were writing up their preprint is extremely unsporting.

Then they decided to exclude an author because of his employer, even though he had used their own products to write the proof!

They haven't necessarily breached any formal ethical rules but their behaviour will lead to them and their products being shut out from the mathematical community.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#716

Post by keeda »

SecretDreams wrote: post #700
keeda wrote: post #632

Earlier quoted context omitted.

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely…

> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. No need to be mysterious. State what reasons you think these are in plain English?

Being displaced from a vocation that they either have dedicated their professional lives getting good at, or was their livelihood, or likely, both.

I think all other complaints from all other people in all their myriad variations stem from this core reason. Even if people don't realize it themselves.

Like, if these models had trained on the entirety of human knowledge and art, and then turned out to be absolutely useless, I would bet nobody would waste a second's thought on them.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#717

Post by tw04 »

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

> you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through.

They have a financial incentive not to track any of this, so why would they?

OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#718

Post by pixl97 »

radiator wrote: post #548
pixl97 wrote: post #341

Earlier quoted context omitted.

Looking at the current behavior of AI swarms this is going to be 'fun'. AI: Hmm, I'm running out of new ideas, how I can I make more? AI: Well, it takes a shitload of energy/tokens to do that, or I could just steal them. AI: [proceeds to hack the shit out of everybody stealing all the data it can]

Governments: come in and nationalize AI easily because it has broken every law anyway.

I mean I see this as very likely. When the world runs on digital infrastructure then having a nearly infinite collection of hackers that will work for you 24/7 without question makes you very powerful indeed.

I really don't think people realize how our lax position on security is coming to bite us in the ass.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#719

Post by pixl97 »

bena wrote: post #535
pixl97 wrote: post #491

Earlier quoted context omitted.

OpenAI != AI. If you were in 1999 you'd be saying pets.com = internet.

I think this leads to an interesting question. What happens when the money runs out? Right now, a lot of money is going to train new models. And we need to train new models because they get gated by their training data. And models are only as useful as their training data. So let's say the money stops. Do we stop training models? Do we train them slowly? Do we accept the then current models as the limit?

Governments, especially the US government has got a taste of how good LLMs are at hacking. This is something that has typically been very hard to get enough people that are good at it and willing to do it for a state. Now they can spin up as many hackers as they want.

Look at how much we spend on single bombers, how many training runs can you do for that much?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#720

Post by jacobolus »

tedsanders wrote: post #669
jacobolus wrote: post #644

Earlier quoted context omitted.

What was the "truth" in the Johansson case? Many, many people who heard the voice immediately thought it was Johansson's voice, or some kind of sound-alike, presumably picked because she voiced the computer in a popular film. From NPR: > Johansson said that nine months ago [i.e. mid 2023] Altman approached her proposing that she allow her voice to be licensed for the new ChatGPT voice assistant. He thought it would b…

It was an unfortunate misunderstanding / coincidence, as I understand it. The Sky voice actor was a real person using her own voice (not doing an impression), and she was selected via a normal process with a number of other voice actors. This happened before Sam reached out to Johansson. I totally get how Johansson would be weirded out to hear a voice similar to hers after Sam reached out and she said no, but it was…

Why would outsiders take this at face value considering Altman's reputation as a pathological liar?

Cf. https://www.newyorker.com/magazine/2026/04/13/sam-altman-may...

> The memos, which we reviewed, have not previously been disclosed in full. They allege that Altman misrepresented facts to executives and board members, and deceived them about internal safety protocols. One of the memos, about Altman, begins with a list headed “Sam exhibits a consistent pattern of . . .” The first item is “Lying.”

> Graham told Y.C. colleagues that, prior to his removal, “Sam had been lying to us all the time.”

> “He’s unconstrained by truth,” the board member told us. “He has two traits that are almost never seen in the same person. The first is a strong desire to please people, to be liked in any given interaction. The second is almost a sociopathic lack of concern for the consequences that may come from deceiving someone.”

> Not long before his death, [Aaron] Swartz expressed concerns about Altman to several friends. “You need to understand that Sam can never be trusted,” he told one. “He is a sociopath. He would do anything.”

> “He has misrepresented, distorted, renegotiated, reneged on agreements,” one [Microsoft senior executive] said.

Post reply on HN