Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

321–330 of 844 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#321

Post by mirsadm »

ueieh wrote: post #140
nirava wrote: post #104

Earlier quoted context omitted.

This has been my line of thinking as well. I have developed a sort of paranoia when I'm working using AI on my projects. Who's to say Claude or OpenAI isn't using the final conclusion of all my ideas, trial and error, and adding it to their database of insights to be offered to the next subscriber for a price? They have demonstrated both the intelligence at scale and the lack of morals for this to not be a problem at…

In the short run it’s fantastic if it means that folks will feed in enough inputs from a wide array of software that can eventually replicate software with smaller teams than historically. Why? Competition. In the long run imagination will win out. No firm has the divine right to exist - it must earn its existence. What OAI and Anthropic have shown is they can accumulate all the information in the world - they still…

They consume everybody's hard work then sell it to all competitors. What a deal.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#322

Post by wrvn »

afzalive wrote: post #44
b800h wrote: post #34

I'm genuinely surprised that more people - including this mathematician in particular - don't untick the "improve the model for everyone" box. Unless the suggestion is that OpenAI ignore this preference?

That doesn't stop them from training on your data apparently. I have that disabled but still has to disable "Don't train on my data" in the privacy center too. https://privacy.openai.com/policies?modal=take-control

Is this claim based on anything besides there being an alternative way to disable it? The privacy center mirrors multiple other functions as well, like account deletion and downloading personal data, but the corresponding buttons in ChatGPT are still doing what they are supposed to.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#323

Post by merksittich »

sashank_1509 wrote: post #266

Both things can be true: 1. OpenAI when using your chats in pretraining is improving its model’s intuition. The model parameter size is massive, and while the data is OOM larger it is plausible that model remembers stuff about chats that improves its latent representation. 2. During RL on verifiable math and massive compute, the model discovers techniques and connections to solve math problems that are superhuman and…

Even OpenAI's own publication [0] on Navier-Stokes from two days ago appears to contradict "basically solving anything you throw at it". The chart shows a pass rate of ~0.5 (vs. Astra's ~0.2) on "a curated set of open math problems". (Based on the timelines and events described in the publication, I presume that the "Internal Model" in the publication represents OpenAI's latest and greatest model. Evidently, this pass rate may improve in the future.)

[0] https://openai.com/index/navier-stokes-solution/

Re: More questions about whether researchers can trust OpenAI with unpublished math

#325

Post by OneManyNone »

derangedHorse wrote: post #310
robotpepi wrote: post #296

Earlier quoted context omitted.

It's in OpenAI's first announcement that they had solved the problem.

> Only after learning the secret to cracking the problem did they send the first prompt. Which quote in the announcement post provides evidence for the above quote?

“ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.”

- https://openai.com/index/navier-stokes-solution/

They do not explicitly admit to knowing about NS specifically, but are extremely explicit that they tried to scoop some potential millennium prize winners.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#326

Post by jsLavaGoat »

amelius wrote: post #318
glitchc wrote: post #295

Earlier quoted context omitted.

The pudding is in the proof. The field is mathematics, the proof can be rigorously verified. If there is a flaw, OpenAI is out to lunch. If the proof is valid, OpenAI has produced something new.

Did you read what they said? The question is now if OAI produced something new or just stole the researchers' good ideas.

Name one discovery ever that didn't depend on someone else's work.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#327

Post by paulsutter »

sashank_1509 wrote: post #266

Both things can be true: 1. OpenAI when using your chats in pretraining is improving its model’s intuition. The model parameter size is massive, and while the data is OOM larger it is plausible that model remembers stuff about chats that improves its latent representation. 2. During RL on verifiable math and massive compute, the model discovers techniques and connections to solve math problems that are superhuman and…

The big question is whether OpenAI is training on "de-identified" sessions that are marked as "do not use for training"

The answer is almost certainly yes, and this is a problem for most users.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#328

Post by [deleted] »

iamgopal wrote: post #292
dgellow wrote: post #286

Earlier quoted context omitted.

I feel that we don’t praise Lean enough. AFAIU it’s what enables LLMs to brute force those problems

True, but could humans cross pollinating lean x prolog x A* ( or any search algorithm) could have solved such math problems with super computer ?

[deleted]

Re: More questions about whether researchers can trust OpenAI with unpublished math

#329

Post by mzs »

ndiddy wrote: post #314
HarHarVeryFunny wrote: post #303

Earlier quoted context omitted.

OpenAI said they sicced this agent army on Navier-Stokes on Sept 1st, while only a couple of days earlier OpenAI's Noam Brown happened to reply to a tweet saying that they had already tried to solve all the Millennium Prize problems and failed... So, it seems either the previous attempt didn't have the training to succeed, or was just not given the compute to do so. Once OpenAI heard that Navier-Stokes was solved, th…

> What we don't know is just how recent this model was, and therefore what it may have been trained on. OpenAI's statement says that they began training their new model on August 28.

omitting when training concluded

edit: ffsm8 makes a great point below, it doesn't matter. I'm not great with dates, sorry.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#330

Post by alansaber »

defmacr0 wrote: post #285
calvbak wrote: post #226

Earlier quoted context omitted.

I always thought that due to the big batch size in SGD/Adam/Muon the model will not memorize a single conversation when trained on, but idk how true that is. The idea of AI companies pin-pointing users that do novel scientific research and then tracking their activity is the direction this points to. I hope that's not the case; that would be bad.

They're almost certainly pin-pointing high-quality conversations and giving them a special weighting. Seems stupid to not do that.

Oh they for sure classify conversations by type (cybersecurity, other guardrail proximates?) and quality.
Post reply on HN