Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Navier–Stokes Millennium Prize Problem

openai.com

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: On the Navier–Stokes Millennium Prize Problem

#741

Post by Imnimo »

tedsanders wrote: post #624
Imnimo wrote: post #254

Earlier quoted context omitted.

>we did not read any private chats The question I am interested in is not "did we read private chats", but "was this new model trained using any of Tristan and Levent's chats, regardless of whether they were marked private". Can you comment on that?

If they opted out of training, then we definitely did not train on them. If they did not opt out, then I don't personally know if training signals came from their chats, and I don't think we'd be able to tell without their cooperation in identifying them. And even if signals were trained on in some manner, I highly doubt it made a difference to a problem as challenging as the NS proof. Reasons for my doubt: - I know…

I don't think you'd lie about it, I don't think you'd train on them if they opted out, and it seems very plausible that this wouldn't have been decisive in whether the model could solve the problem. That said, it also seems at least possible that a key idea or a particular step found its way into training data. It wouldn't mean OpenAI stole their proof - clearly the model developed its own approach.

Either way, it seems worth having clarity, and I'm a bit surprised OpenAI's stance is just "we can't rule this out, but don't worry about it". OpenAI is, apparently, very happy to use unreleased models to try to scoop big results if they get a whiff that someone else is close (which strikes me as pretty scummy regardless of any issues of training contamination). It seems like people who might want to use OpenAI's models as part of their research would want to be very clear about whether doing so can make them, even in principle, more likely to fall victim to this.

Re: On the Navier–Stokes Millennium Prize Problem

#742

Post by tzone »

irthomasthomas wrote: post #702
tedsanders wrote: post #41

Earlier quoted context omitted.

Yes, that was the allegation last night. I work at OpenAI, though not on the team that did this, and my understanding is: - we decided to ask our model for Millenium problem solutions because of two reasons: (a) our new model was looking incredibly good and (b) we heard rumors that some Millenium problems had been solved and were curious if our models could solve them (the goal here was not to scoop any particular in…

If the goal was not to scoop them, why did openai put a massive team on this, working weekends, only after they heard rumors of the solution?

Clearly The goal was to scoop Anthropic not a single researcher. OpenAI heard the rumor that Anthropic solved an open problem. So they went nuts pulling all plugs to scoop them.

Turns out it wasn’t actually Anthropic and just a researcher with a single Anthropic guy friend working on it .

Wild times

Re: On the Navier–Stokes Millennium Prize Problem

#743

Post by aeve890 »

sp527 wrote: post #730
redox99 wrote: post #598

Earlier quoted context omitted.

The amount of goalpost moving is insane. "Yeah it can solve Millenium problems, but can it do it with nothing more than a one sentence prompt?" Also there are proofs where the only human steering was "keep going".

Well, in fairness, the OP asked for counterpoints. He didn't stipulate that they need to be reasonable.

Extraordinary claims require extraordinary evidence. If OP claim superhuman genius, then they should prove superhuman genius. Simple as that.

Re: On the Navier–Stokes Millennium Prize Problem

#744

Post by sp527 »

rakejake wrote: post #147
paxys wrote: post #37

Earlier quoted context omitted.

What else can they declare really? Yeah the model has training data from previous attempts. Alpöge and Buckmaster also similarly benefited from attempts before theirs.

I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's co…

> Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget.

This is a bad argument. This is clearly not how it works. And unless Tristan is some truly alien-like savant (and maybe he is), what's necessary to initiate the AI's work already exists in countless published research papers and not exclusively in his head or notes. AIs can survey the sum total of all prior work on a problem and discern reasonable paths for inquiry.

Tristan is acting as if he's working off of an outdated model of AI, similar to primitive chess-playing models that winnowed the search space much more deterministically. If someone this intelligent truly doesn't get that this is not at all what AI is anymore, then maybe there's no hope that we ever understand it.

But I think he does realize this and he's flailing about for counterarguments from a place of bitterness and dejection, accepting even those that are too weak to be defensible. And that is very human and even forgivable.

Re: On the Navier–Stokes Millennium Prize Problem

#745

Post by yauneyz »

intenex wrote: post #440

I think this is clear evidence that AI models are now at the far frontier of mathematics innovation and discovery and exceed human limits. This specific problem having had a $1 million bounty on its head and still remaining unsolved for 26 years after the bounty was placed is pretty clear evidence that many of the world's best human mathematicians would have solved this problem if they could have, and none were able…

I think the biggest hurdle remaining is that all these landmark results are generally counter-examples.

Proving something in the affirmative often requires the creation of an entire new sub-field of math, or new tools. Think of Fermat's Last Theorem or something like that.

These results, while impressive, are clever constructions using existing techniques. It isn't clear that AIs can build new machinery like this. But if/when they can, yeah it is probably game over.

Re: On the Navier–Stokes Millennium Prize Problem

#746

Post by tristanj »

vemacs wrote: post #718
Jtariiiii wrote: post #607

Earlier quoted context omitted.

>Just because something is legal and permitted by terms of service doesn't mean it's morally right. What are you expecting OpenAI to do exactly if these mathematicians voluntarily submitted their prompts into ChatGPT's training data? Are they supposed to manually review all their data to make sure competing mathematicians didn't accidentally leave the "submit prompts" toggle on? Or were they supposed to not try to so…

> Are they supposed to manually review all their data t Yes. They should determine if training data included this teams data. Consider the money they spent, the press release and the purpose of their publication. Since they failed to answer this question they shouldn't have published.

[dead]

Re: On the Navier–Stokes Millennium Prize Problem

#747

Post by ndriscoll »

lambda wrote: post #218
rfgplk wrote: post #156

Earlier quoted context omitted.

> Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data? Probably? I have a few hundred TB of training data for various small scale models and I can attest that I have _no idea_ what's in them. As in, literally zero. Half is scraped from GitHub and other hosting sites, other than that, I couldn't tell you anything else. At OpenAI's scale their entire pipeline is likel…

Yeah, I'm sure it's completely automated. But that doesn't preclude being able to index and track what the sources of data are. For your data sets, I would hope you are including source information for where the data came frome. And at OpenAI's scale, I would presume they are doing some amount of rolling hashing or similar to weed out duplication, training on too much duplicate data can cause problems. AllenAI have a…

Attributing training data seems pointless for trustworthiness. The way you trust a model is the same way you trust a human; you ask it to:

  1. Provide a chain of reasoning from agreed premises. These days LLMs can even do this airtight with proof assistants.

  2. Cite data sources for non-agreed premises. I don't care where the model learned a fact. It might not have ever read a document directly from the primary source. I want it to link directly to either widely agreed facts (e.g. standard textbooks, and if necessary school syllabi demonstrating that the text is standard) or primary sources (e.g. datasets).
Training provenance is irrelevant. It's neither necessary nor sufficient to deal with truth.

Re: On the Navier–Stokes Millennium Prize Problem

#748

Post by tzone »

concinds wrote: post #511
Eridrus wrote: post #466

Earlier quoted context omitted.

Seeing mathematicians such as Terry Tao being unhappy with open problems being solved makes me sort of question the usefulness of any of this pure mathematics. If we're not happy that the problems are being solved, why care about this field at all?

His issue is more nuanced than that. Most of the value was in humans reaching new insights or new math during failed attempts to solve these problems, whereas AI is basically "too efficient" in beelining to the goal and discards potential new insights reached along the way. I assume this is solvable.

Well, if in future we do end up with a magical tool that can solve any formal mathematical problem on a whim, we really won’t need field of mathematics anymore as it is today.

There would be no need to deliver new mathematical insights by solving problems. You would just have a magical math problem solving machine and that’s it.

Re: On the Navier–Stokes Millennium Prize Problem

#749

Post by ninjahawk1 »

The problem is the precedent this creates. For non-famous people using public APIs like this it could mean AI companies sucking up the information and throwing millions in compute at it.

The sequence for Navier-Stokes was that these researcher spent a year working on it, then they published a possible breakthrough, OpenAI then spends $15M within a couple days to finish it.

This was incredibly opportunistic.

Re: On the Navier–Stokes Millennium Prize Problem

#750

Post by FabCH »

tantalor wrote: post #369
echelon wrote: post #363

Earlier quoted context omitted.

There's an event horizon and we're maybe past it?

Heh. Wrong "singularity"

They meant that even the happens-very-fast AI singularity isn’t sub-picosecond. It still exists in time, and has a duration.

The event horizon would then be the time period between the singularity becoming inevitable and it actually happening.

Post reply on HN