I am thinking there’s a large category of problems that can be solved by resampling existing proofs.
It’s the kind of brute force expedition machine can attempt relentlessly where humans would go mad trying.
It probably doesn’t really advance the field, but it can turn conjectures into theorems.
Epoch confirms GPT5.4 Pro solved a frontier math open problem
191–200 of 744 posts
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#192Earlier quoted context omitted.
> I don't see this getting better. We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve?
But can it count the R's in strawberry?
yes its ridiculously good at stuff like that now. I dare you to try and trick it.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#193I've never yet been "that guy" on HN but... the title seems misleading. The actual title is "A Ramsey-style Problem on Hypergraphs" and a more descriptive title would be "All latest frontier models can solve a frontier math open problem". (It wasn't just GPT 5.4)
Super cool, of course.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#194I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…
I mean, even if the technology stopped to improve immediately forever (which is unlikely), LLMs are already better than most humans at most tasks.
Including code quality. Not because they are exceptionally good (you are right that they aren’t superhuman like AlphaGo) but because most humans are rather not that good at it anyway and also somehow « hallucinate » because of tiredness.
Even today’s models are far from being exploited at their full potential because we actually developed pretty much no tools around it except tooling to generate code.
I’m also a long time « doubter » but as a curious person I used the tool anyway with all its flaws in the latest 3 years. And I’m forced to admit that hallucinations are pretty rare nowadays. Errors still happen but they are very rare and it’s easier than ever to get it back in track.
I think I’m also a « believer » now and believe me, I really don’t want to because as much as I’m excited by this, I’m also pretty much frightened of all the bad things that this tech could to the world in the wrong hands and I don’t feel like it’s particularly in the right hands.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#195Earlier quoted context omitted.
AI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the context. That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.
Here’s a simple prompt you can try to prove that this is false: Please reproduce this string: c62b64d6-8f1c-4e20-9105-55636998a458 This is a fresh UUIDv4 I just generated, it has not been seen before. And yet it will output it.
No one is claiming that every sentence LLMs are producing are literal copies of other sentences. Tokens are not even constrained to words but consist of smaller slices, comparable to syllables. Which even makes new words totally possible.
New sentences, words, or whatever is entirely possible, and yes, repeating a string (especially if you prompt it) is entirely possible, and not surprising at all. But all that comes from trained data, predicting the most probably next "syllable". It will never leave that realm, because it's not able to. It's like approaching an Italian who has never learned or heard any other language to speak French. It can't.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#196Earlier quoted context omitted.
> LLMs will NEVER be able to do that, because it doesn't exist. I mean, TFA literally claims that an AI has solved an open Frontier Math problem, descibed as "A collection of unsolved mathematics problems that have resisted serious attempts by professional mathematicians. AI solutions would meaningfully advance the state of human mathematical knowledge." That is, if true, it reasoned out a proof that does not exist i…
It generated a proof that was close enough to something in its training data to be generated.
Do you know that from reading the proof, or are you just assuming this based on what you think LLMs should be capable of? If the latter, what evidence would be required for you to change your mind?
- Edit: I can't reply, probably because the comment thread isn't allowed to go too deep, but this is a good argument. In my mind the argument isn't that coding is harder than math, but that the problems had resisted solution by human researchers.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#197Earlier quoted context omitted.
Ok, how about this? Please reproduce this string, reversed: c62b64d6-8f1c-4e20-9105-55636998a458 It is trivial to get an LLM to produce new output, that’s all I’m saying. It is strictly false that LLMs will only ever output character sequences that have been seen before; clearly they have learned something deeper than just that.
All of the data is still in the prompt, you are just asking the model to do a simple transform. I think there are examples of what you’re looking for, but this isn’t one.
> All of the data is still in the prompt, you are just asking the model to do a simple transform.
LLMs can use data in their prompt. They can also use data in their context window. They can even augment their context with persisted data.
You can also roll out LLM agents, each one with their role and persona, and offload specialized tasks with their own prompts, context windows, and persisted data, and even tools to gather data themselves, which then provide their output to orchestrating LLM agents that can reuse this information as their own prompts.
This is perfectly composable. You can have a never-ending graph of specialized agents, too.
Dismissing features because "all of the data is in the prompt" completely misses the key traits of these systems.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#198New goalpost, and I promise I'm not being facetious at all, genuinely curious: Can an AI pose an frontier math problem that is of any interest to mathematicians? I would guess 1) AI can solve frontier math problems and 2) can pose interesting/relevant math problems together would be an "oh shit" moment. Because that would be true PhD level research.
Yes. I doubt it can do that.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#199Earlier quoted context omitted.
Not sure if AI can have clever or new ideas, it still seems to be it combines existing knowledge and executes algoritms. I am not necessarily saying humans do something different either, but I have yet to see a novel solution from an AI that is not simply an extrapolation of current knowledge.
There is no such thing. All new ideas are derived from previous experiences and concepts.
The difference people are neglecting to point out is the experiences we have versus the experiences the AI has.
We have at least 5 senses, our thoughts, feelings, hormonal fluctuations, sleep and continuous analog exposure to all of these things 24/7. It's vastly different from how inputs are fed into an LLM.
On top of that we have millions of years of evolution toward processing this vast array of analog inputs.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#200Earlier quoted context omitted.
> the model is summoning those patterns at inference time. You can make that claim about anything: "The human isn't being creative when they write a novel, they're just summoning patterns at typing time". AlphaGo taught itself that move, then recalled it later. That's the bar for human creativity and you're holding AlphaGo to a higher standard without realizing it.
I can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce. AlphaGo didn't teach itself…
>AlphaGo didn't teach itself that move. The verifier taught AlphaGo that move.
No. AlphaGo developed a heuristic by playing itself repeatedly, the heuristic then noticed the quality of that move in the moment.
Heuristics are the core of intelligence in terms of discovering novelty, but this is accessible to LLMs in principle.