Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

571–580 of 850 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#571

Post by famouswaffles »

HarHarVeryFunny wrote: post #563
famouswaffles wrote: post #539

Earlier quoted context omitted.

1. I would agree if the rumours were that some mathematician(s) had solved them, but the rumors alleged it was Anthropic. I don't really see what the big deal was. They had a new model that was going along great and wanted to test its mettle. 2. Yes Brubeck's comments were weird at face value. That said, Open AI's proof isn't a duplication of anything. Not only is Tristan's work a sub problem but the methods are diff…

> What sort of guidance do you think is happening in a 10k agent, 320b token, 88 hour run ? AI did this one If you read the PDF release by Buckmaster, apparently the initial claim from Brubeck was that there as very little human input involved, then as the call progressed more and more people popped up that has been involved with it. Does this aspect really matter? Not really, other than OpenAI wanting to present thi…

>If you read the PDF release by Buckmaster, apparently the initial claim from Brubeck was that there as very little human input involved, then as the call progressed more and more people popped up that has been involved with it.

As it seems and as they tell it, they started the run modestly and diverted more resources towards it as it looked more and more promising. The run didn't start with 10k agents for instance. The point is there isn't anything humans are doing in this timeframe against all this text that would count more than "little human output". It's still a fair assessment I would say.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#572

Post by alexgoodhart »

luma wrote: post #567
dwaltrip wrote: post #513

Earlier quoted context omitted.

I heard they also tried to strong-arm them into removing the name of their collaborator who happened to work at a different company (Anthropic)... I haven't looked into it myself, but if true, that seems incredibly scummy.

The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons. What a mess.

I think people are too reserved in their unwillingness to operationalize ambiguity. Ambiguity is constantly being thrown in our face, with internal audits and other laughable attestations of virtue that amount to a pantomime of transparency / good faith.

Why should I care if a company claims they find no evidence of wrongdoing? Is that the threshold for privacy/trust? “We don’t care if it appears that we’ve been dishonest unless there’s hard proof.” They can simply design proof keeping to terminate at the places their dishonesty is implemented.

For me, when there is a clear motive to be dishonest, a corporation should be assumed to be dishonest unless there are robust transparency measures and a regulatory environment shown to be providing a cost to dishonesty. Without it, all you do is burden yourself while the powerful entity moves ahead with its selective dishonesty and the rewards there reaped.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#573

Post by jltsiren »

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

I think it's better to ignore OpenAI here, because OpenAI didn't do anything.

Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards.

Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question.

Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities).

And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#574

Post by nautikos2 »

Most people here are missing the forest for the trees.

We live in a society where phones and internet providers and websites all collect an incredible amount of data about everywhere you go, what you do, and what you think. In the US, we have very few digital rights.

We are building a society where a trillion dollar company can aggregate all this data and just yoink your shiny new idea away from you at the finish line.

This is double plus ungood.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#575

Post by phatfish »

tedsanders wrote: post #505
nsagent wrote: post #492

Earlier quoted context omitted.

Can you speak to why in both cases, the problems OpenAI's models solved used the same techniques the mathematicians were exploring, which also happened to be niche approaches to the problem. As an NLP researcher myself, I find that coincidence highly suspect unless the models focused most of their attempts on the predominant approaches (they are trained for MLE after all).

I'm not a mathematician and I don't want to speculate about anything I can't back up. All I know about Navier-Stokes is from my graduate fluid dynamics class at Stanford a decade ago (where I received a poor grade). However, I don't want to leave you hanging, so what I will say is: - I've heard some people say the model's solution is quite different from theirs (but I have no clue how to personally assess the spiritu…

OK bro.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#576

Post by HarHarVeryFunny »

famouswaffles wrote: post #559
suddenlybananas wrote: post #549

Earlier quoted context omitted.

>Brubeck's comments were weird at face value This is an odd way to gloss over threats.

I put it like that because of Brubeck's own words on the matter. You're acting like we've gotten email receipts here. I'm not really interested in going over a he-said she-said about strangers.

Brubeck has admitted what he said, but claims he immediately retracted it as a "poor choice of words".

Given Buckmaster's telling, this seems beyond "poor choice of words"... It was a veiled threat, that he then doubled down on with his "If you don’t want me to be nice, then I don’t have to be nice." follow-up.

**

I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”

**

FWIW there are also other people on Twitter, such as this DeepMind researcher, saying this is a pattern for Brubeck.

https://x.com/dheeraj_nagaraj/status/2097266146445774924?s=2...

Re: More questions about whether researchers can trust OpenAI with unpublished math

#577

Post by cnity »

brumbelow wrote: post #432
gnfargbl wrote: post #111

Earlier quoted context omitted.

That link is a helpful contribution to this discussion. I'm not at all familiar with this area, but my reading is that he appears to call it out as a relatively obvious extension of his own work: > It is a creative and at the same time elementary construction that uses not just property (T) for an application of my result with Kun, but also for the ambient group G in order to overcome the problem, that the Γ-componen…

> On the other side, I was looking myself for such a mechanism ever since we wrote the paper in 2019 and admire the efficiency of this construction. He seems to admit very clearly he does not see this as his own work. 'I was looking...' well why did he stop? Because the AI figured it out first. It seems quite odd to me to 'admire the construction' of something, only for your opinion to sour once that something figure…

> It seems quite odd to me to 'admire the construction' of something, only for your opinion to sour once that something figures it out first.

Not to be too cute here, but this is like every artistic rivalry ever.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#579

Post by augment_me »

mlazos wrote: post #18

It’s crazy to me that companies/researchers share important data with these AI labs, you’re basically giving them your secret sauce which they then share with all of your competitors via training on conversations. At the same time I don’t really know alternatives other than a slightly less than frontier local LLM. Not sure how good they are at math.

LMFTFY:

"Its crazy to me that some people are not egotistical, self-centered, and don't solely care about fame and wealth accumulation".

Re: More questions about whether researchers can trust OpenAI with unpublished math

#580

Post by hellohello2 »

atleastoptimal wrote: post #343

Most scientific breakthroughs are simply a continuation of previous work. I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did. Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforti…

Of previous, not concurrent work. Science is friendly competition, and spying on others is unfriendly.
Post reply on HN