This will be remembered as one of the biggest milestones in AI progress. The drama around it will at best be a footnote, just like hardly anyone caring about the drama around Poincare conjecture today.
I agree. Nobody cares and will care about the drama, it is just marketing. This is the point where were definitely have reached AGI.
To the extent the LLM copied the plagiarized work, no. Like a number of the Erdos problem solutions turned out to be based on forgotten literature. So the drama matters even for the assessment of the mega agent work.
There are lots of startups creating labs that can be managed e2e by agents. That will connect reasoning to the physical world and dramatically speed up the plan, experiment, reflect loop beyond what humans currently do in science R&D.
Maybe. Maybe not. Look at AI drug design - it's not really speeding up the important part - drug trials. There isn't really a coherent plan to use AI for the most complex part of drug discovery at all.
I work in pharma and I've advocated for directly training models that predict drug trial outcomes. I don't think we have the necessary support (data + algorithms) to produce accurate models in this space. I suspect that predicting drug trial outcomes is roughly isomorphic to understanding human biology at a fundamental level (and accepting that our existing models of how drugs work are extremely limited).
Even under your interpretation, OAI pushed a button and solved NS. Yes, that is very impressive. Are you kidding me? Imagine building an automated system that can solve NS.
"I was shown a prompt and told the internal research model had simply been
given the problem statement. Levent had been told by Sebastien “very little
human input” had been used. This turned out not to be true. Over the course
of the call, as members of their team sent Sebastien corrections and details over
their internal chat, it emerged that an entire team had been working on the
problem".
But that lead happened to be the same approach Levent and Tristan had found... Meaning they would have found it first if OpenAI hadn't spent millions in compute on following their lead to its conclusion faster than them.
It's forced vs unforced Euler, so it's not exactly the same. Since OpenAI has access to their training data, they can probably scrub through the data to find out whether there have been any mentions of the similar approach, and whether it only comes from Tristan Buckmaster or if it is in the training data before that. They'll probably have to kick off another fleet of agents to scrub through the training data to answ…
> they can probably scrub through the data to find out whether there have been any mentions of the similar approach
They did check and it was in the trainingset but they could not confirm if it was the /cause/ of it, which is a bit of a cop out.
created a simulation of the solution to describe what's happening and why it's important for engineers, climate modeling, etc : https://navier-stokes-singularity-simulator.netlify.app/ (updated so that it works better on mobile)
Is it really important for engineers, etc?
In reality you can't have things like infinite velocity, so if Navier-Stokes is predicting blowups, then isn't this a problem with Navier-Stokes not handling some edge cases, not a problem with reality that engineers need to be concerned about?
They managed to solve a problem that was beyond current human ability.
That was the net effect (assuming what they solved was the actual problem and not a loophole in the problem statement or a lean bug). My point is they need not all work coherently to do that -- for example, for all we know 3/4 of them went off the rails, their results were pruned, and the relevant results came from a random subset that happened to produce something useful.
How can you possible say this sort of thing in context of what looks like a millenium prize being solved. I swear there's nobody blinder than those who won't see.
Because it seems like most of the work may have been done by human mathematicians and cribbed by OpenAI at the last minute
The human mathematicians didn't solve the Navier-Stokes problem, they solved the Euler problem. And they were extensively using LLMs to drive the work, as described in the Buckmaster statement.
Any way you cut it, this is a major achievement for AI, besotted with human drama over whose prompt should be recognized by the history books.
> If you need privacy, then you are going to have to pay full price for those tokens (API). At this point, how can we even trust that they aren't accidentally training on those tokens too?
It'd be corporate suicide for them to be caught violating zero-data-retention commitments. But also if you're really paranoid you can just use ChatGPT on Azure or AWS, where nothing is flowing back to OpenAI at all.
Zero data retention commitments? Technically generating synthetic data off of user data on the fly, counts as zero retention of user data! And it technically means they didn't lie either, because they didn't keep user data, they kept their own, synthetic data. That is what they already do with reasoning when they provide summaries/sanitized versions of it to prevent "distillation attacks" even via the API. They generate synthetic data aka summaries of reasoning traces, and that is technically not user data, because the user did not enter it directly into the LLM! It still counts as zero data retention when they only keep original LLM created reasoning traces as well as all LLM outpit, because the user did not enter it, so it counts as synthetic data and is thus not user data.
It'd be corporate suicide for them to be caught violating zero-data-retention commitments. But also if you're really paranoid you can just use ChatGPT on Azure or AWS, where nothing is flowing back to OpenAI at all.
They have no zero data retention commitments anymore though? In fact they have quite the opposite, a direct message of storing prompts since the release of sol (but we wont train on it trust us wink wink)
Not storing prompts but they absolutely can store reasoning traces, summarized or sanitized reasoning traces and the whole LLM output as well.