Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the Navier–Stokes Millennium Prize Problem

openai.com

Post reply on HN
Print view View original

991–1000 of 1001 posts

Re: On the Navier–Stokes Millennium Prize Problem

#991

Post by JumpCrisscross »

dekhn wrote: post #983
JumpCrisscross wrote: post #893

Earlier quoted context omitted.

> We don't have any proof of that It’s fair to give benefit of doubt to Buckmaster given OpenAI is currently being very credibly sued by Apple for openly stealing others’ original work in another context.

A prior of "one large company is current involved in an unrelated lawsuit with another large company" is pretty weak; the case is undecided and about an entirely different kind of IP theft. In short I think a lot of people are jumping to conclusions without supporting evidence and that's really not helping the situation.

> the case is undecided and about an entirely different kind of IP theft

The guy mocked accessing his prior employer's circuit diagrams and was protected by OpenAI until Apple filed suit.

Tabula rasa, sure, we need more evidence. But ignoring the priors should at least be explicitly acknowledged.

Re: On the Navier–Stokes Millennium Prize Problem

#992

Post by joshka »

tim-kt wrote: post #815
joshka wrote: post #761

Earlier quoted context omitted.

I worked at OpenAI previously, but don't know any of the people involved in this. My guess was it was probably this was more a nerd snipe than any action from OpenAI that was a "massive team" being put on it. Literally someone looking at this and asking "I wonder if our models are good enough yet". It's easy to assume that having access to massive compute amounts means significant coordination, but this assumes that…

They said that a customer would have paid around 15 million for the required compute. I can't imagine that this was not a significant internal spending even with "free" tokens.

At Astra API prices that's 300B tokens (I saw 130B output tokens claimed elsewhere), large but not unheard of if you consider it across a few people doing random experiments with best-of-n type things. On my personal account, I've done a billion+ token days just on a normal pro 20x subscription. I know many others that wildly outpaced that by orders of magnitude. This was apparently 130B over 89 hours, so about 30x that rate. When things are free and you're expected to token max 30x seems fairly reasonable to me.

If you consider this as a cost to be compared against the question: "What does it take to be able to prove that you have a model that can solve the hardest problems that humans know about?", then spending a some amount of thousands/millions to know the boundaries of that seems not too important in comparison.

You've also got to consider this as compute that's allocated to pushing the frontier of what models can do, so while it's using GPUs that have been paid for etc., it's not like it's a cost that's supposed to be use less of this so that others can have capacity. If you made researchers afraid to use capacity like this, a lot of the things that improve would tend to do so significantly slower. (some may say that's a good thing ;)

A good way to think about this is when tokens are free, you get to choose whether you're optimizing for latency or intelligence rather than having to consider price.

---

Publically, tibo (Codex owner) in Feb this year: https://x.com/thsottiaux/status/2024649339344445825

> OpenAI employees currently get unlimited inference. Usage is now peaking at > XX billion tokens per week for some of them.

Mathew Berman (AI Youtuber) in Jun: https://x.com/MatthewBerman/status/2067270730795134984

> I've used 25 billion tokens in the last 7 days.

Re: On the Navier–Stokes Millennium Prize Problem

#993

Post by pama »

topaz0 wrote: post #969
pama wrote: post #929

Earlier quoted context omitted.

They managed to solve a problem that was beyond current human ability.

That was the net effect (assuming what they solved was the actual problem and not a loophole in the problem statement or a lean bug). My point is they need not all work coherently to do that -- for example, for all we know 3/4 of them went off the rails, their results were pruned, and the relevant results came from a random subset that happened to produce something useful.

If you work with distributed systems, you still call that scenario a success. On the other hand, if the 3/4 of agents going off the rails bring down the whole mission, that is a failure. The latter would have been my guess with current models scaling to 10k agents.

I am not an expert in lean4, but I could follow parts of the high level lean definitions of the problem statement in the repo. A lean bug would be a fun scenario; I am certain this proof will receive the deserved scrutiny, and if it uncovers a bug, it will make the story even more exciting. It is extremely unlikely to be the case, however, because the 10k agents working on the proof didnt use lean, so it would have to be a math logic error that translates to a lean bug—perhaps something the agents picked up during training?

Re: On the Navier–Stokes Millennium Prize Problem

#994

Post by pama »

danielmarkbruce wrote: post #697
pama wrote: post #183

Earlier quoted context omitted.

Not only that, but it used 10k agents coherently over 88 hours to come up with the proof. This is a significant advance.

If you can create a graph of independent work, which you can with many such problems, agents can work together nicely. Again, thank Lean and the tooling around it.

As far as I understand the 10k agents worked on the proof. The lean formalization came later and was easier/faster than getting the proof.

Re: On the Navier–Stokes Millennium Prize Problem

#995

Post by pama »

HarHarVeryFunny wrote: post #954
pama wrote: post #183

Earlier quoted context omitted.

Not only that, but it used 10k agents coherently over 88 hours to come up with the proof. This is a significant advance.

With 10,000 agents and $20M of compute this is just brute force search. It's a bit like telling 10,000 kids there's an easter egg hidden over there, pointing to one corner of your yard (or having "heard a rumor" it was hidden in that corner). If you have $20M to spend on your problem, then yes, AI brute force search is an option, but unless you know a solution is possible (as OpenAI did here), you may still be wastin…

You jest and that is OK. Brute force search is not something you can do over math problems of that difficulty or anything with combinatorial complexity.

To me it feels closer to taking the top 10k human mathematicians on a large retreat for a year and having them self organize to collectively solve this problem—not kids and easter eggs.

Re: On the Navier–Stokes Millennium Prize Problem

#996

Post by jdm2212 »

mmanfrin wrote: post #887
jdm2212 wrote: post #726

Earlier quoted context omitted.

It'd be corporate suicide for them to be caught violating zero-data-retention commitments. But also if you're really paranoid you can just use ChatGPT on Azure or AWS, where nothing is flowing back to OpenAI at all.

> It'd be corporate suicide for them to be caught violating zero-data-retention commitments Would it, though? Considering their entire business model is built on the agglomeration of data that isnt theirs.

Their business model is based on making money from LLMs. That is some mix of (a) selling API access to businesses and (b) making money from ads / referral fees / whatever from consumer usage.

The businesses API usage will disappear overnight if OpenAI doesn't honor its contracts with businesses.

Re: On the Navier–Stokes Millennium Prize Problem

#997

Post by aizk »

simianwords wrote: post #136
aizk wrote: post #52

People had joked a couple years ago "Well if they solve a Millenium problem it's AGI"... Well here we are.

> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon https://news.ycombinator.com/item?id=38433655 > Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief t…

Did you just happen to have all those comments saved? Quite the timing.

Re: On the Navier–Stokes Millennium Prize Problem

#998

Post by runtime_terror »

Can I ask a nieve question?

Why would OpenAI volunteer to give credit / share authorship if they believe they indeed solved the problem independently and in a different way?

Based on how they seem to operate in general, this smells more like CYA than it does being honorable, but that's my bias.

Re: On the Navier–Stokes Millennium Prize Problem

#999

Post by tedsanders »

morleytj wrote: post #868
tedsanders wrote: post #181

Earlier quoted context omitted.

To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effec…

Thanks for the details, it's definitely believable, but if the user had not consented to have their conversations used for training, then shouldn't it be straightforward to state that their conversations were never used for training? If you need to do a whole series of extensive experiments to check in that scenario, it implies there are pathways for your conversations to end up in training even though you opted out…

Yep, if they opted out then we didn't train on them.

My comment was about the scenario where they didn't opt out. In that case, it's possible that a droplet of their data went into the ocean of other training data, and it's very difficult to measure what effect that droplet had. My expectation is essentially zero impact, but no one can know for sure.

Re: On the Navier–Stokes Millennium Prize Problem

#1000

Post by tedsanders »

Chance-Device wrote: post #643
tedsanders wrote: post #181

Earlier quoted context omitted.

To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effec…

Please answer this question: do you or do you not train your models on anonymized user data, where those users have opted out of such training? The blog post appears to imply the answer to this is yes, as otherwise I assume it would be impossible for this contamination to have happened.

If someone has opted out, then we don't train on their data.
Post reply on HN