I have a dumb feeling that the proof will be wrong with serious flaws but that will be found out only after the ipo
On the Navier–Stokes Millennium Prize Problem
861–870 of 1001 posts
Re: On the Navier–Stokes Millennium Prize Problem
#862"we cannot rule out that de-identified data derived from their usage of our products helped improve our models ." What a landmine sentence to bury in this report, you can't rule out your models were spying on other researchers?
If they had agreed to let OpenAI train on their data, it wouldn’t be spying.
In the academic world it would still be deeply problematic…pick your preferred word.
An analogy is akin to reviewing a paper. If I review a paper with some novel findings and then use my massive lab of graduate students to do the obvious next step before the other paper makes it through type setting and then shove it out as a pre print, I didn’t win - I was a jerk.
There are lots of cases of people using peer review or other accesss to efectively forerun others work and get credit. It’s a known problem of the nature of knowledge validation in academia, it’s not solved and it’s not deterministic but people know it when they see it.
Re: On the Navier–Stokes Millennium Prize Problem
#863Earlier quoted context omitted.
Yeah well, its easy to do if you steal someone elses work and then try to threaten them into staying quiet about it Edit: OpenAI have now admitted they were training on prompts at the time they made their breakthrough: https://mastodon.social/@tristanbuckmaster/11723647135247030...
Steal someone elses work, whose work was also AI generated . . .
[dead]
Re: On the Navier–Stokes Millennium Prize Problem
#864created a simulation of the solution to describe what's happening and why it's important for engineers, climate modeling, etc : https://navier-stokes-singularity-simulator.netlify.app/ (updated so that it works better on mobile)
Re: On the Navier–Stokes Millennium Prize Problem
#865Earlier quoted context omitted.
In OpenAI's case, if they were genuinely unsure, they wouldn't have said anything. "We cannot rule out" means they absolutely 100% for-sure did look at the existing prompts and bootstrapped from that, and they are trying to get ahead of the disclosure with this weasel-wording.
Also possible: we're 99.999% sure, but a lawyer said to be safe and strictly accurate, we should stick in a sentence in saying we can't be perfectly sure, since it's infeasible for us to prove it. I promise you that if we took their work from ChatGPT and stuck in a bunch of weasel words to give the opposite impression while remaining technically true, I would quit on the spot. (I work at OpenAI.)
While you can't necessarily prove it, you can say whether the data was in the training set at all.
You can also do something like a release of a GPT-OSS v2, where you actually release training data and checkpoints, and do an experiment where you have some held out math problem dataset, then demonstrate how much training it takes on solutions (or partial solutions) to that dataset before the model saturates that test. While of course that would be a test on a much smaller model, it would cost a tiny fraction of the training on your big model, and it could be used to demonstrate just how much effect data contaminaiton like this could have, especially if you did the same experiment on a few different sized of model to show the scaling laws involved.
Re: On the Navier–Stokes Millennium Prize Problem
#866Earlier quoted context omitted.
I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's co…
> Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget. This is a bad argument. This is clearly not how it works. And unless Tristan is some truly alien-like savant (and maybe he is), what's necessary to initiate the A…
Are you saying there is no search space intractable to LLMs? That wouldn't be possible. AIs are statistical pattern-matchers on steroids. The prompt is key to getting anything useful out of them. They are incredibly useful and major game changers but ultimately that does not alter this fact. People (including OAI) have already tried to solve Millenium Problems with it. That OAI woke up last week and suddenly decided that throwing their researchers armed with millions of compute on one particular idea to a problem is highly suspicious in itself.
Even if OAI had zero data from Buckmaster's sessions, this is in very poor taste and highly unethical. You are front running a researcher just to be able to say you did it first? Tao is right - OAI is treating math results like oil. This is the like Exxon getting a whiff of a massive oil field and racing to the punch by deploying their full crew.
Re: On the Navier–Stokes Millennium Prize Problem
#867Earlier quoted context omitted.
The allegations of contamination (using Tristan and Levent's work) aren't very well evidenced, but this behavior by OpenAI (from the authors' statement) makes them seem like the bad guys: > I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin…
I´m waiting on the other side version, because I know there is no justifiable way to talk to a person like they did. Sociopathic behaviour.
Talking like that and threatening an academic like that is crazy. I read the explanations Altman and the others posted and they completely skip over the whole "I don't have to be nice" style threats.
Re: On the Navier–Stokes Millennium Prize Problem
#868My take: 1. It shows what even this wave of AI can actually do. 2. I wish it were done by different folks, ideally under some kind of public control like NASA research or the NPR model. 3. Keep in mind: natural science is different. It's not always a matter of computation. Computer science folks often struggle with this -- but this virtual world here does not actually exist. Everything is physical, including informat…
The problem with physics and chemistry is that you need simulations and those are often in themselves compute hungry. So the iteration loop will be slower.
Although there are companies trying to work around that too, from PhysicsX to some of the world model co’s.
Re: On the Navier–Stokes Millennium Prize Problem
#869Earlier quoted context omitted.
Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data? This is one of the major problems with these enormous closed models, and even most open-weights models, which don't disclose their training process or training data. You can never be sure what went into its training. Did it come up with an idea originally, or is it just plagiarising its training data? Are there ma…
To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effec…
Thanks for the details, it's definitely believable, but if the user had not consented to have their conversations used for training, then shouldn't it be straightforward to state that their conversations were never used for training?
If you need to do a whole series of extensive experiments to check in that scenario, it implies there are pathways for your conversations to end up in training even though you opted out of that setting.
Of course, this is assuming that the toggle was set to not consent to training. I can't know that of course, but if this is considered a possibility even after using an enterprise account or toggling off data retention, it's a bit concerning.
Re: On the Navier–Stokes Millennium Prize Problem
#870In CS speak very roughly this would mean something like disproving an algorithm by giving it a case that fails it. Right?