Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Navier–Stokes Millennium Prize Problem

openai.com

Post reply on HN
Print view View original

861–870 of 1001 posts

Re: On the Navier–Stokes Millennium Prize Problem

#861

Post by lambda »

tedsanders wrote: post #357
Analemma_ wrote: post #129

Earlier quoted context omitted.

In OpenAI's case, if they were genuinely unsure, they wouldn't have said anything. "We cannot rule out" means they absolutely 100% for-sure did look at the existing prompts and bootstrapped from that, and they are trying to get ahead of the disclosure with this weasel-wording.

Also possible: we're 99.999% sure, but a lawyer said to be safe and strictly accurate, we should stick in a sentence in saying we can't be perfectly sure, since it's infeasible for us to prove it. I promise you that if we took their work from ChatGPT and stuck in a bunch of weasel words to give the opposite impression while remaining technically true, I would quit on the spot. (I work at OpenAI.)

While you can't necessarily prove it, you can say whether the data was in the training set at all.

You can also do something like a release of a GPT-OSS v2, where you actually release training data and checkpoints, and do an experiment where you have some held out math problem dataset, then demonstrate how much training it takes on solutions (or partial solutions) to that dataset before the model saturates that test. While of course that would be a test on a much smaller model, it would cost a tiny fraction of the training on your big model, and it could be used to demonstrate just how much effect data contaminaiton like this could have, especially if you did the same experiment on a few different sized of model to show the scaling laws involved.

Re: On the Navier–Stokes Millennium Prize Problem

#862

Post by rakejake »

sp527 wrote: post #739
rakejake wrote: post #147

Earlier quoted context omitted.

I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's co…

> Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget. This is a bad argument. This is clearly not how it works. And unless Tristan is some truly alien-like savant (and maybe he is), what's necessary to initiate the A…

Are you saying there is no search space intractable to LLMs? That wouldn't be possible. AIs are statistical pattern-matchers on steroids. The prompt is key to getting anything useful out of them. They are incredibly useful and major game changers but ultimately that does not alter this fact. People (including OAI) have already tried to solve Millenium Problems with it. That OAI woke up last week and suddenly decided that throwing their researchers armed with millions of compute on one particular idea to a problem is highly suspicious in itself.

Even if OAI had zero data from Buckmaster's sessions, this is in very poor taste and highly unethical. You are front running a researcher just to be able to say you did it first? Tao is right - OAI is treating math results like oil. This is the like Exxon getting a whiff of a massive oil field and racing to the punch by deploying their full crew.

Re: On the Navier–Stokes Millennium Prize Problem

#863

Post by morleytj »

igleria wrote: post #184
colinhb wrote: post #122

Earlier quoted context omitted.

The allegations of contamination (using Tristan and Levent's work) aren't very well evidenced, but this behavior by OpenAI (from the authors' statement) makes them seem like the bad guys: > I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin…

I´m waiting on the other side version, because I know there is no justifiable way to talk to a person like they did. Sociopathic behaviour.

Talking like that and threatening an academic like that is crazy. I read the explanations Altman and the others posted and they completely skip over the whole "I don't have to be nice" style threats.

Re: On the Navier–Stokes Millennium Prize Problem

#864

Post by m11a »

geremiiah wrote: post #370
hdivider wrote: post #112

My take: 1. It shows what even this wave of AI can actually do. 2. I wish it were done by different folks, ideally under some kind of public control like NASA research or the NPR model. 3. Keep in mind: natural science is different. It's not always a matter of computation. Computer science folks often struggle with this -- but this virtual world here does not actually exist. Everything is physical, including informat…

The problem with physics and chemistry is that you need simulations and those are often in themselves compute hungry. So the iteration loop will be slower.

Although there are companies trying to work around that too, from PhysicsX to some of the world model co’s.

Re: On the Navier–Stokes Millennium Prize Problem

#865

Post by morleytj »

tedsanders wrote: post #181
lambda wrote: post #130

Earlier quoted context omitted.

Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data? This is one of the major problems with these enormous closed models, and even most open-weights models, which don't disclose their training process or training data. You can never be sure what went into its training. Did it come up with an idea originally, or is it just plagiarising its training data? Are there ma…

To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effec…

Thanks for the details, it's definitely believable, but if the user had not consented to have their conversations used for training, then shouldn't it be straightforward to state that their conversations were never used for training?

If you need to do a whole series of extensive experiments to check in that scenario, it implies there are pathways for your conversations to end up in training even though you opted out of that setting.

Of course, this is assuming that the toggle was set to not consent to training. I can't know that of course, but if this is considered a possibility even after using an enterprise account or toggling off data retention, it's a bit concerning.

Re: On the Navier–Stokes Millennium Prize Problem

#867

Post by angry_octet »

jdm2212 wrote: post #844
angry_octet wrote: post #836

Earlier quoted context omitted.

Everyone?? No, most definitely OpenAI. But they have learnt their lesson, next time they won't reach out to who they stole it from, they will publish first.

Seems like OpenAI did a boring normal corporate thing (find out your competitor made a breakthrough, try to replicate it) and then when the other mathematicians found out OpenAI had beat them to Navier-Stokes, they decided to lie about what happened because they were upset they didn't get to make the big breakthrough themselves.

You are entitled to your opinion, but it isn't supported by the information already available.

Re: On the Navier–Stokes Millennium Prize Problem

#868

Post by sp527 »

rakejake wrote: post #862
sp527 wrote: post #739

Earlier quoted context omitted.

> Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget. This is a bad argument. This is clearly not how it works. And unless Tristan is some truly alien-like savant (and maybe he is), what's necessary to initiate the A…

Are you saying there is no search space intractable to LLMs? That wouldn't be possible. AIs are statistical pattern-matchers on steroids. The prompt is key to getting anything useful out of them. They are incredibly useful and major game changers but ultimately that does not alter this fact. People (including OAI) have already tried to solve Millenium Problems with it. That OAI woke up last week and suddenly decided…

> Even if OAI had zero data from Buckmaster's sessions, this is in very poor taste and highly unethical. You are front running a researcher just to be able to say you did it first? Tao is right - OAI is treating math results like oil. This is the like Exxon getting a whiff of a massive oil field and racing to the punch by deploying their full crew.

I want to be clear that I agree with this view and with Tao more generally. But we're all just yelling at the wind now.

Re: On the Navier–Stokes Millennium Prize Problem

#869

Post by [deleted] »

Chance-Device wrote: post #641
tedsanders wrote: post #181

Earlier quoted context omitted.

To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effec…

Please answer this question: do you or do you not train your models on anonymized user data, where those users have opted out of such training? The blog post appears to imply the answer to this is yes, as otherwise I assume it would be impossible for this contamination to have happened.

[deleted]

Re: On the Navier–Stokes Millennium Prize Problem

#870

Post by waterTanuki »

tedsanders wrote: post #41
pavel_lishin wrote: post #8

Is this the one that was allegedly based on someone else's actual work & prompts? https://news.ycombinator.com/item?id=49605915 https://bsky.app/profile/quantian.bsky.social/post/3muyhwbcd... https://cims.nyu.edu/~tristanb/statement.pdf

Yes, that was the allegation last night. I work at OpenAI, though not on the team that did this, and my understanding is: - we decided to ask our model for Millenium problem solutions because of two reasons: (a) our new model was looking incredibly good and (b) we heard rumors that some Millenium problems had been solved and were curious if our models could solve them (the goal here was not to scoop any particular in…

> - the proof generated by our model was very different from theirs and also goes far beyond the published literature

I'm hearing two completely conflicting stories. Buckmaster is claiming the approach used by OpenAI is so strikingly similar to the one he used, that mere coincidence is astronomically small. Yet OpenAI is claiming that the methods used are entirely different.

Anyone care to provide primary evidence proving one way or the other?

Post reply on HN