Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

511–520 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#511

Post by jacquesm »

TeMPOraL wrote: post #500
Yizahi wrote: post #473

Earlier quoted context omitted.

It probably can, but won't realize that and it won't be efficient in that. LLM can shuffle tokens for an enormous number of tries and eventually come up with something super impressive, though as you yourself have mentioned, we would need to have a mandatory verification loop, to filter slop from good output and how to do it outside of some limited areas is a big question. But assuming we have these verification loop…

> We never had a big demand to define how humans are intelligent or conscious etc, since it is too hard and was relegated to a some frontier researchers. And with LLMs we now do have such demand but the science wasn't ready. So we are all collectively searching in the dark, trying to define if we are different from these programs if not how. I certainly can't do that. I do know that LLMs are useful, but I also suspec…

> (It's also why "hallucinations" and "prompt injection" are not bugs, but fundamental facets of what makes LLMs useful. They cannot and will not be "fixed", any more than you can "fix" humans to be immune to confabulation and manipulation. It's just the nature of fully general sytems.)

This is spot on and one of the reasons why I don't think putting LLMs or LLM based devices into anything that requires security is a good idea.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#512

Post by famouswaffles »

datsci_est_2015 wrote: post #504
famouswaffles wrote: post #494

Earlier quoted context omitted.

You made a pretty nonsensical argument, pretty much seems like the big standard for these arguments. What does linear regression have to do with the limitations of a stacked transfer ? Absolutely nothing. This is the problem here. You don't know shit and just make up whatever. You can see people doing the same thing in GPT-1, 2, 3, 4 threads all telling us why LLMs will never be able to do thing it manages to do late…

> You don’t know shit lol. Why so emotionally charged? Are you perhaps worried that you’ve invested too much time and effort into a technology that may not deliver what influencers have been promising for years? Like a proverbial bagholder? > What does linear regression have to do with the limitations of a stacked transfer ? Absolutely nothing. This is the problem here. We’re talking about fundamental concepts of mod…

>We’re talking about fundamental concepts of modeling in this subthread. LLMs, despite what influencers may tell you, are simply models. I’ll even throw you a bone and admit they are models for intelligence. But they are still models, and therefore all of the things that we have learned about “models” since Plato are still relevant. Most importantly, since Plato we’ve known that “models” have fundamental limits vs. what they try to represent, otherwise they would be a facsimile, not a model.

Okay, but the brain is also “just a model” of the world in any meaningful sense, so that framing does not really get you anywhere. Calling something a model does not, by itself, establish a useful limit on what it can or cannot do. Invoking Plato here just sounds like pseudo-profundity rather than an actual argument.

>I hope you enjoy winning these imaginary arguments against these imaginary comments. The fundamental limitations of LLMs discussed since GPT-1 have never been addressed by changing the architecture of the underlying model. All of the improvements we’ve experienced have been due to (1) improvements in training regime and (2) harnesses / heuristics (e.g. Agents).

If a capability appears once training improves, scale increases, or better inference-time scaffolding is added, then it was not demonstrated to be a 'fundamental impossibility'.

That is the core issue with your argument: you keep presenting provisional limits as permanent ones, and then dressing that up as theory. A lot of people have done that before, and they have repeatedly been wrong.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#513

Post by jacquesm »

LatencyKills wrote: post #401
stavros wrote: post #393

Earlier quoted context omitted.

Huh, Claude one-shotted it out of a single message from me. Man, LLMs have gotten good.

No it didn't. Like I said... it may have gotten something that worked but there is no way Claude got it to work while supporting multi-spaces, multi-desktops, and using under 2% cpu utilization. My solution can display app window content even when those windows are minimized, which is not something the content server supports. My point was that Claude realized all the SKC problems and came up with a solution that 99%…

And now it does.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#514

Post by Capricorn2481 »

59nadir wrote: post #158
bandrami wrote: post #135

It's deeply surprising to me that LLMs have had more success proving higher math theorems than making successful consumer software

Software developers have spent decades at this point discounting and ignoring almost all objective metrics for software quality and the industry as a whole has developed a general disregard for any metric that isn't time-to-ship (and even there they will ignore faster alternatives in favor of hyped choices). (Edit: Yes, I'm aware a lot of people care about FP, "Clean Code", etc., but these are all red herrings that d…

This doesn't pass a sniff test. We have plenty of ways to verify good software, else you wouldn't be making this post. You know what bad software is and looks like. We want something fast that doesn't throw an error every 3 page navigations.

You can ask an LLM to make code in whatever language you want. And it can be pretty good at writing efficient code, too. Nothing about NPM bloat is keeping you from making a lean website. And AI could theoretically be great at testing all parts of a website, benchmarking speeds, trying different viewports etc.

But unfortunately we are still on the LLM train. It just doesn't have anything built-in to do what we do, which is use an app and intuitively understand "oh this is shit." And even if you could allow your LLM to click through the site, it would be shit at matching visual problems to actual code. You can forget about LLMs for true frontend work for a few years.

And they are just increasingly worse with more context, so any non-trivial application is going to lead to a lot of strange broken artifacts, because text prediction isn't great when you have numerous hidden rules in your application.

So as much as I like a good laugh at failing software, I don't think you can blame shippers for this one. LLMs are not struggling in software development because they are averaging a lot of crap code, it's because we have not gotten them past unit tests and verifying output in the terminal yet.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#515

Post by staticassertion »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

I think "novel" is ill defined here, perhaps. LLMs do appear to be poor general reasoners[0], and it's unclear if they'll improve here.

It would be unintuitive for them to be good at this, given that we know exactly how they're implemented - by looking at text and then building a statistical model to predict the next token. From this, if we wanted to commit to LLMs having generalizable knowledge, we'd have to assume something like "general reasoning is an emergent property of statistical token generation", which I'm not totally against but I think that's something that warrants a good deal of evidence.

A single math problem being solved just isn't rising to that level of evidence for me. I think it is more on you to:

1. Provide a theory for how LLMs can do things that seemingly go beyond expectations based on their implementation (for example, saying that certain properties of reasoning are emergent or reduce to statistical constructs).

2. Provide evidence that supports your theory and ideally can not be just as well accounted for another theory.

I'm not sure if an LLM will never generate "novel" content because I'm not sure that "novel" is well defined. If novel means "new", of course they generate new content. If novel means "impressive", well I'm certainly impressed. If "novel" means "does not follow directly from what they were trained on", well I'm still skeptical of that. Even in this case, are we sure that the LLM wasn't trained on previous published works, potentially informal comments on some forum, etc, that could have steered it towards this? Are we sure that the gap was so large? Do we truly have countless counterexamples? Obviously this math problem being solved is not a rigorous study - the authors of this don't even have access to the training data, we'd need quite a bit more than this to form assumptions.

I'm willing to take a position here if you make a good case for it. I'm absolutely not opposed to the idea that other forms of reasoning can't reduce to statistical token generation, it just strikes me as unintuitive and so I'm going to need to hear something to compel me.

[0] https://jamesfodor.com/2025/06/22/line-goes-up-large-languag...

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#516

Post by LatencyKills »

jacquesm wrote: post #513
LatencyKills wrote: post #401

Earlier quoted context omitted.

No it didn't. Like I said... it may have gotten something that worked but there is no way Claude got it to work while supporting multi-spaces, multi-desktops, and using under 2% cpu utilization. My solution can display app window content even when those windows are minimized, which is not something the content server supports. My point was that Claude realized all the SKC problems and came up with a solution that 99%…

And now it does.

Sorry, "now it does", what?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#517

Post by graemep »

mikkupikku wrote: post #497
datsci_est_2015 wrote: post #336

Earlier quoted context omitted.

I’ve seen this style of take so much that I’m dying for someone to name a logical fallacy for it, like “appeal to progress” or something. Step away from LLMs for a second and recognize that “Yesterday it was X, so today it must be X+1” is such a naive take and obviously something that humans so easily fall into a trap of believing (see: flying cars).

Logical fallacies are vastly overrated. Unless the conversation is formal logic in the first place, "logical fallacies" are just a way to apply quick pattern matching to dismiss people without spending time on more substantive responses. In this case, both you and the other are speculating about the near future of a thing, neither of you knows.

OK, its not a logical fallacy, its a false assumption.

The belief in the inevitability of progress is a bad assumption. Especially if you assume a particular technology will keep advancing.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#520

Post by hackinthebochs »

datsci_est_2015 wrote: post #460
hackinthebochs wrote: post #443

Earlier quoted context omitted.

Sure, that's true as well. But I don't see this as a substantive response given that the only people making unsupported claims in this thread are those trying to deflate LLM capabilities.

So, to review this thread - OP asked for someone to make a logical argument for the separation of “training” from “model” - I made the argument - You cherry picked an argument against my specific example and made an appeal to emergent complexity - I pointed out that emergent complexity isn’t limitless - “the only people making unsupported claims in this thread are those trying to deflate LLM capabilities”

To be clear, you are confusing me with other commenters in this thread. All I want is for those that liken LLMs to stochastic parrots and other deflationary claims to offer an argument that engages with the actual structure of LLMs and what we know about them. No one seems to be up to that challenge. But then I can't help but wonder where people's confident claims come from. I'm just tired of the half-baked claims and generic handwavy allusions that do nothing but short-circuit the potential for genuine insight.
Post reply on HN