Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

441–450 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#441

Post by datsci_est_2015 »

stavros wrote: post #411
datsci_est_2015 wrote: post #409

Earlier quoted context omitted.

> Talking about how they find it hard to say they aren't sure of something is a much more interesting limitation to talk about, for example. Sure, thank you for steelmanning my argument. I didn’t think I needed to actually spell out all of the fundamental limitations of LLMs in this specific thread. They are spoken at length across the web, but are often met with pushback, which was my entire point. Here’s another on…

But that's also like saying "humans don't have a memory property, any 'memory' is in the hippocampus". It's not useful to say that "an LLM you don't bother to keep training has no memory". Of course it doesn't, you removed its ability to form new memories!

So why then do we stop training LLMs and keep them stored at a specific state? Is it perhaps because the results become terrible and LLMs have a delicate optimal state for general use? This sounds like an even worse case for a model of intelligence.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#442

Post by fsflover »

SequoiaHope wrote: post #344
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

Most inventions are an interpolation of three existing ideas. These systems are very good at that.

I can't even find a good example of an invention that is not an interpolation.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#443

Post by hackinthebochs »

datsci_est_2015 wrote: post #440
hackinthebochs wrote: post #429

Earlier quoted context omitted.

Linear regression has well characterized mathematical properties. But we don't know the computational limits of stacked transformers. And so declaring what LLMs can't do is wildly premature.

> And so declaring what LLMs can't do is wildly premature. The opposite is true as well. Emergent complexity isn’t limitless. Just like early physicists tried to explain the emergent complexity of the universe through experimentation and theory, so should we try to explain the emergent complexity of LLMs through experimentation and theory. Specifically not pseudoscience, though.

Sure, that's true as well. But I don't see this as a substantive response given that the only people making unsupported claims in this thread are those trying to deflate LLM capabilities.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#444

Post by stavros »

datsci_est_2015 wrote: post #441
stavros wrote: post #411

Earlier quoted context omitted.

But that's also like saying "humans don't have a memory property, any 'memory' is in the hippocampus". It's not useful to say that "an LLM you don't bother to keep training has no memory". Of course it doesn't, you removed its ability to form new memories!

So why then do we stop training LLMs and keep them stored at a specific state? Is it perhaps because the results become terrible and LLMs have a delicate optimal state for general use? This sounds like an even worse case for a model of intelligence.

Nope, it's not that, but it's nice of you to offer a straw man. Makes the argument flow better.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#445

Post by al_borland »

stavros wrote: post #343
benterix wrote: post #325

Earlier quoted context omitted.

I'm very happy to say calculators are far better than me in calculations (to a given precision). I'm happy to admit computers are so much better than me in so many aspects. And I have problem saying LLMs are very helpful tools able to generate output so much better than mine in almost every field of knowledge. Yet, whenever I ask it to do something novel or creative, it falls very short. But humans are ingenious beas…

But the question isn't whether you can get LLMs to do something novel, it's whether anyone can get them to do something novel. Apparently someone can, and the fact that you can't doesn't mean LLMs aren't good for that.

When it comes to LLMs doing novel things, is it just the infinite monkey theorem[0] playing out at an accelerated rate, helped along by the key presses not being truly random?

Surely if we tell the LLM to do enough stuff, something will look novel, but how much confirmation bias is at play? Tens of millions of people are using AI and the biggest complaint is hallucinations. From the LLMs perspective, is there any difference between a novel solution and a hallucination, other than dumb luck of the hallucination being right?

[0] https://en.wikipedia.org/wiki/Infinite_monkey_theorem

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#446

Post by stavros »

al_borland wrote: post #445
stavros wrote: post #343

Earlier quoted context omitted.

But the question isn't whether you can get LLMs to do something novel, it's whether anyone can get them to do something novel. Apparently someone can, and the fact that you can't doesn't mean LLMs aren't good for that.

When it comes to LLMs doing novel things, is it just the infinite monkey theorem[0] playing out at an accelerated rate, helped along by the key presses not being truly random? Surely if we tell the LLM to do enough stuff, something will look novel, but how much confirmation bias is at play? Tens of millions of people are using AI and the biggest complaint is hallucinations. From the LLMs perspective, is there any dif…

This argument doesn't go the way you want it to go. Billions of people exist, but maybe a few tens of thousands produce novel knowledge. That's a much worse rate than LLMs.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#448

Post by mike_hearn »

zurfer wrote: post #310

"In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh)." I find that very surprising. This problem seems out of reach 3 months ago but now the 3 frontier models are able to solve it. Is everybody distilling each others models? Companies sell the same data and RL environment to all big labs? Anybody more involved can share some rumors? :P I d…

Yes there's a whole ecosystem of companies that create and sell RL gyms to AI labs and of course they develop their own internally too. You don't hear much about this ecosystem because RL at scale is all private. Nearly no academic research on it.

A lot of this is probably just throwing roughly equal amounts of compute at continuous RLVR training. I'm not convinced there's any big research breakthrough that separates GPT 5.4 from 5.2. The diff is probably more than just checkpoints but less than neural architecture changes and more towards the former than the latter.

I think it's just easy to underestimate how much impact continuous training+scaling can have on the underlying capabilities.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#449

Post by jacquesm »

KellyCriterion wrote: post #342
jacquesm wrote: post #306

Earlier quoted context omitted.

> e.g. 167,383 * 426,397 = 71,371,609,051 They may be wrong, but so are you.

No, its correct: https://www.google.com/search?q=167383+*+426397

You missed the point.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#450

Post by [deleted] »

hackinthebochs wrote: post #200
energy123 wrote: post #122

Earlier quoted context omitted.

I can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce. AlphaGo didn't teach itself…

>AlphaGo didn't teach itself that move. The verifier taught AlphaGo that move. No. AlphaGo developed a heuristic by playing itself repeatedly, the heuristic then noticed the quality of that move in the moment. Heuristics are the core of intelligence in terms of discovering novelty, but this is accessible to LLMs in principle.

[deleted]
Post reply on HN