Epoch confirms GPT5.4 Pro solved a frontier math open problem • Hacker News Board

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#661

Post by acchow » Wed, Mar 25, 2026, 4:23 AM UTC

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

> asserting that LLMs will never generate 'truly novel' ideas or problem solutions

I don't think I've had one of these my entire life. Truly novel ideas are exceptionally rare:

- Darwin's origin of the species - Godel's Incompleteness - Buddhist detachment

Can't think of many.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#662

Post by markatto » Wed, Mar 25, 2026, 4:27 AM UTC

zamalek wrote: post #656

luma wrote: post #654

Earlier quoted context omitted.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

This is special pleading. Long multiplication is a trivial form of reasoning that is taught at elementary level. Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper. That was Opus with extended reasoning, it had all the opportunity to get it right, but didn't. There are people who can quickly multiply such…

I tried this with Claude - it has to be explicitly instructed to not make an external tool call, and it can get the right answer if asked to show its work long-form.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#663

Post by nearbuy » Wed, Mar 25, 2026, 4:28 AM UTC

energy123 wrote: post #456

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

> 67,383 * 426,397 = 71,371,609,051 ... You need to say why it can do some novel tasks but could never do others. Model interpretability gives us the answers. The reason LLMs can (almost) do new multiplication tasks is because it saw many multiplication problems in its training data, and it was cheaper to learn the compressed/abstract multiplication strategies and encode them as circuits in the network, rather than m…

> This gives it the ability to approximate multiplication problems it hasn't seen before.

More than approximate. It straight up knows the algorithms and will do arbitrarily long multiplications correctly. (Within reason. Obviously it couldn't do a multiplication so large the reasoning tokens would exceed its context window.)

Having ChatGPT 5.4 do 1566168165163321561 * 115616131811365737 without tools, after multiplying out a lot of coefficients, it eventually answered 181074305022287409585376614708755457, which is correct.

At this point, it's less misleading to say it knows the algorithm.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#664

Post by jhanschoo » Wed, Mar 25, 2026, 4:30 AM UTC

scoofy wrote: post #600

qnleigh wrote: post #597

Earlier quoted context omitted.

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

I studied philosophy focusing on the analytic school and proto-computer science. LLMs are going to force many people start getting a better understanding about what "Knowledge" and "Truth" are, especially the distinction between deductive and inductive knowledge. Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. In the empirical wor…

I think we may have similar perspectives. Regarding empirical knowledge, consider when the knowledge is in relation to chaotic systems. Characterize chaotic systems at least as systems where inaccurate observations about the system in the past and present while useful for predicting the future, nevertheless see the errors grow very quickly for the task of predicting a future state. Then indeed, prediction is difficult.

One domain of knowledge I think you have yet to mention. We can talk about fundamentally computationally hard problems. What comes to mind regarding such problems that are nevertheless of practical benefit are physics simulations, material simulations, fluid simulations, but there exist problems that are more provably computationally difficult. It seems to me that with these systems, the chaotic nature is one where even if you have one infinitely precise observation of a deterministic system, accessing a future state of the system is difficult as well, even though once accessed, memorization seems comparatively trivial.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#665

Post by esafak » Wed, Mar 25, 2026, 4:34 AM UTC

zamalek wrote: post #656

luma wrote: post #654

Earlier quoted context omitted.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

This is special pleading. Long multiplication is a trivial form of reasoning that is taught at elementary level. Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper. That was Opus with extended reasoning, it had all the opportunity to get it right, but didn't. There are people who can quickly multiply such…

Mathematics is not the only kind of reasoning, so your conclusion is false. The human brain also has compartments for different types of activities. Why shouldn't an AI be able to use tools to augment its intelligence?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#666

Post by CamperBob2 » Wed, Mar 25, 2026, 4:40 AM UTC

zamalek wrote: post #656

luma wrote: post #654

Earlier quoted context omitted.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

This is special pleading. Long multiplication is a trivial form of reasoning that is taught at elementary level. Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper. That was Opus with extended reasoning, it had all the opportunity to get it right, but didn't. There are people who can quickly multiply such…

Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper

LOL, talk about special pleading. Whatever it takes to reshape the argument into one you can win, I guess...

LLMs don't reason.

Let's see you do that multiplication in your head. Then, when you fail, we'll conclude you don't reason. Sound fair?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#667

Post by mrtesthah » Wed, Mar 25, 2026, 4:44 AM UTC

PxldLtd wrote: post #307

EternalFury wrote: post #191

I am thinking there’s a large category of problems that can be solved by resampling existing proofs. It’s the kind of brute force expedition machine can attempt relentlessly where humans would go mad trying. It probably doesn’t really advance the field, but it can turn conjectures into theorems.

I'm of the opinion that everything we've discovered is via combinatorial synthesis. Standing on the shoulders of giants and all that. I'm not sure I've seen any convincing argument that we've discovered anything ex nihilo.

How about this guy? https://en.wikipedia.org/wiki/Srinivasa_Ramanujan

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#668

Post by ares623 » Wed, Mar 25, 2026, 4:50 AM UTC

luma wrote: post #655

famouswaffles wrote: post #109

Earlier quoted context omitted.

Last I checked humans didn't pop into existence doing that. It happened after billions of years of brute force, trial and error evolution. So well done for falling into the exact same trap the OP cautions. Intelligence from scratch requires a mind boggling amount of resources, and humans were no different.

And then an 18-to-20-something-year training run is required for each individual instance.

I know right, such a waste. Plus it's so random on how they will turn out!

Any suggestions on how to reduce that waste?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#669

Post by mrtesthah » Wed, Mar 25, 2026, 4:53 AM UTC

zurfer wrote: post #310

"In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh)." I find that very surprising. This problem seems out of reach 3 months ago but now the 3 frontier models are able to solve it. Is everybody distilling each others models? Companies sell the same data and RL environment to all big labs? Anybody more involved can share some rumors? :P I d…

Maybe so, but GPT 5.4 is absolutely pulling ahead. You can see the differences visually on https://minebench.ai/.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#670

Post by whyenot » Wed, Mar 25, 2026, 5:12 AM UTC

snemvalts wrote: post #76

virgildotcodes wrote: post #56

I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. It's this pervasive belief that underlies so much discussion around what it means to be intelligent. The null hypothesis goes out the window. People constantly make comments like "well it's just trying a bunch of stuff until something works" and it seems that they do not pause for a moment to consider whethe…

The ability to learn and infer without absorbing millions of books and all text on internet really does make us special. And only at 20 watts!

Just an interesting thought experiment: if you took all the sensory information that a child experiences through their senses (sight, hearing, smell, touch, taste) between, say, birth and age five, how many books worth of data would that be? I asked Claude, and their estimate was about 200 million books. Maybe that number is off ± by an order of magnitude. ...but then again Claude is only three years old, not five.