Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

651–660 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#651

Post by zamalek »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3

> Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631

Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939

> 2 165 842 392 930 662 831

Your example seems short enough to not pose a problem.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#652

Post by peacebeard »

zamalek wrote: post #651
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

This doesn’t address the author’s point about novelty at all. You don’t need 100% accuracy to have the capability to solve novel problems.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#653

Post by drfloyd51 »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

People rarely create things that are wholly new.

Most created things are remixes of existing things.

Hallucinations are “something new”. And like most new things, useless. But the truth is the entire conversation is a hallucination. We just happen to agree that most of it is useful.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#654

Post by luma »

zamalek wrote: post #651
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#655

Post by luma »

famouswaffles wrote: post #109
snemvalts wrote: post #76

Earlier quoted context omitted.

The ability to learn and infer without absorbing millions of books and all text on internet really does make us special. And only at 20 watts!

Last I checked humans didn't pop into existence doing that. It happened after billions of years of brute force, trial and error evolution. So well done for falling into the exact same trap the OP cautions. Intelligence from scratch requires a mind boggling amount of resources, and humans were no different.

And then an 18-to-20-something-year training run is required for each individual instance.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#656

Post by zamalek »

luma wrote: post #654
zamalek wrote: post #651

Earlier quoted context omitted.

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

This is special pleading.

Long multiplication is a trivial form of reasoning that is taught at elementary level. Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper. That was Opus with extended reasoning, it had all the opportunity to get it right, but didn't. There are people who can quickly multiply such numbers in their head (I am not one of them).

LLMs don't reason.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#657

Post by [deleted] »

aaroninsf wrote: post #616
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

Ximm's Law applies ITT: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. Especially the lemmas: - any statement about AI which uses the word "never" to preclude some feature from future realization is false. - contemporary implementations have almost always already been improved upon, but are unevenly distributed.

[deleted]

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#658

Post by lern_too_spel »

zamalek wrote: post #651
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

This hasn't been true for a while now.

I asked Gemini 3 Thinking to compute the multiplication "by hand." It showed its work and checked its answer by casting out nines and then by asking Python.

Sonnet 4.6 with Extended Thinking on also computed it correctly with the same prompt.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#659

Post by radio879 »

scoofy wrote: post #600
qnleigh wrote: post #597

Earlier quoted context omitted.

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

I studied philosophy focusing on the analytic school and proto-computer science. LLMs are going to force many people start getting a better understanding about what "Knowledge" and "Truth" are, especially the distinction between deductive and inductive knowledge. Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. In the empirical wor…

Also, we can do thought experiments, simulations in our heads, that often are as good as doing them for real - it has limitations and isn't perfect though. But it does work often. Einstein used to purposely dose off in a weird position so that something hit his leg or something like that to slightly nudge him half awake so he could remember his half-dreaming state - which is where he discovered some things

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#660

Post by t-writescode »

EternalFury wrote: post #191

I am thinking there’s a large category of problems that can be solved by resampling existing proofs. It’s the kind of brute force expedition machine can attempt relentlessly where humans would go mad trying. It probably doesn’t really advance the field, but it can turn conjectures into theorems.

I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future. Other people in I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future.

I only ever learned it in school, but if memory serves, Prolog is a whole "given these rules, find the truth" sort of language, which aligns well with these sorts of problem spaces. Mix and match enough, especially across disparate domains, and you might get some really interesting things derived and discovered that are low-hanging fruit just waiting to be discovered.

Post reply on HN