Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

521–530 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#521

Post by aspenmartin »

Yizahi wrote: post #470
ggamezar wrote: post #467

Earlier quoted context omitted.

> It doesn't discern between them, just looks for the best statistical fit. Why this is not true for humans?

We can't tell yet if that is true, partially true, or false for humans. We do know that LLM can't do anything else besides that (I mean as a fundamental operating principle).

Why is it important? “Statistical fit” is what you want…not understanding this is indicative of a limited understanding of what statistics is. What do you think it means to truly understand something? I don’t get it: read probability theory by Jaynes. It doesn’t really matter if the brain does Bayesian updates but that’s what’s optimal…

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#522

Post by adam_arthur »

energy123 wrote: post #456
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

> 67,383 * 426,397 = 71,371,609,051 ... You need to say why it can do some novel tasks but could never do others. Model interpretability gives us the answers. The reason LLMs can (almost) do new multiplication tasks is because it saw many multiplication problems in its training data, and it was cheaper to learn the compressed/abstract multiplication strategies and encode them as circuits in the network, rather than m…

Which is exactly how humans learn many things too.

E.g. observing a game being played to form an understanding of the rules, rather than reading the rulebook

Or: Observing language as a baby. Suddenly you can speak grammatically correctly even if you can't explain the grammar rules.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#523

Post by aspenmartin »

gf000 wrote: post #479
LatencyKills wrote: post #475

Earlier quoted context omitted.

Unless you worked on the macOS content server directly you’d have no idea that my solution was even possible. That fact that Claude skipped over all the obvious solutions is why I used the word novel.

How confident are you that this knowledge was not part of the training data? Was there no stackoverflow questions/replies with it, no tech forum posts, private knowledge bases, etc? Not trying to diminish its results, just one should always assume that LLMs have a rough memory on pretty much the whole of the internet/human knowledge. Google itself was very impressive back then in how it managed to dig out stuff inter…

Do you think AlphaGo is regurgitating human gameplay? No it’s not: it’s learning an optimal policy based on self play. That is essentially what you’re seeing with agents. People have a very misguided understanding of the training process and the implications of RL in verifiable domains. That’s why coding agents will certainly reach superhuman performance. Straw/steel man depending on what you believe: “But they won’t be able to understand systems! But a good spec IS programming!” also a bad take: agents absolutely can interact with humans, interpret vague deseridata, fill in the gaps, ask for direction. You are not going to need to write a spec the same way you need to today. It will be exactly like interacting with a very good programmer in EVERY sense of the word

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#524

Post by adam_arthur »

I don't understand the position that learning through inference/example is somehow inferior to a top down/rules based learning.

Humans learn many, and perhaps even the majority, of things through observed examples and inference of the "rules". Not from primers and top down explanation.

E.g. Observing language as a baby. Suddenly you can speak grammatically correctly even if you can't explain the grammar rules.

Or: Observing a game being played to form an understanding of the rules, rather than reading the rulebook

Further: the majority of "novel" insights are simply the combination of existing ideas.

Look at any new invention, music, art etc and you can almost always reasonably explain how the creator reached that endpoint. Even if it is a particularly novel combination of existing concepts.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#525

Post by ryandvm »

SequoiaHope wrote: post #344
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

Most inventions are an interpolation of three existing ideas. These systems are very good at that.

Indeed. Every time someone complains that LLMs can't come up with anything new, I'm assaulted with the depressing remembrance that neither do I.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#526

Post by LatencyKills »

TeMPOraL wrote: post #519
LatencyKills wrote: post #516

Earlier quoted context omitted.

Sorry, "now it does", what?

The things it didn't, which you then helpfully spelled out.

Do you believe my brief overview of the problem will help Claude identify the specific undocumented functions required for my solution? Is that how you think data gets fed back into models during training?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#527

Post by aspenmartin »

staticassertion wrote: post #515
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

I think "novel" is ill defined here, perhaps. LLMs do appear to be poor general reasoners[0], and it's unclear if they'll improve here. It would be unintuitive for them to be good at this, given that we know exactly how they're implemented - by looking at text and then building a statistical model to predict the next token. From this, if we wanted to commit to LLMs having generalizable knowledge, we'd have to assume…

The “good deal of evidence” is everywhere. The proof is in the pudding. Of course you can find failure modes, the blog article (not an actual paper?) rightfully derides benchmarks and then…creates a benchmark? Designed to elicit failure modes, ok so what? As if this is surprising to anyone and somehow negates everything else?

Anyone who says that “statistical models for next token generation” are unlikely to provide emergent intelligence I think is really not understanding what a statistical model for next token generation really means. That is a proxy task DESIGNED to elicit intelligence because in order to excel at that task beyond a certain point you need to develop the right abstractions and decide how to manipulate them to predict the next token (which, by the way, is only one of many many stages of training). That’s indistinguishable from intelligence. It’s like saying “I think it’s surprising that a jumble of trillions of little cells zapping each other would produce emergent intelligence” while ignoring the fact that brains are clearly intelligent.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#528

Post by wrqvrwvq »

hodgehog11 wrote: post #206
wrqvrwvq wrote: post #119

Earlier quoted context omitted.

It's only because humans came up with a problem, worked with the ai and verified the result that this achievement means anything at all. An ai "checking its own work" is practically irrelevant when they all seem to go back and forth on whether you need the car at the carwash to wash the car. Undoubtedly people have been passing this set of problems to ai's for months or years and have gotten back either incorrect res…

Funding a few PhDs for a year costs orders of magnitude more than it did to solve this problem in inference costs. Also, this has been active research for some time. Or I guess the people working on it are just not as good as a random bunch of students? It's amazing the lengths that people go to maintain their worldview, even if it means belittling hardworking people. I take it you're not a mathematician. This is an…

Inference costs are heavily subsidised. My point was that we've spent trillions collectively on ai, and so far we have a few new proofs. It's been active research but the problem estimates only 5-10 people are even aware that it is a problem. I wrote "math phd's" not "random students", but regardless, I wouldn't know how you interpreted my statement that people could have discovered without ai this as "belittling the people working on this". You seem like a stupid person with an out of control chatbot that can't comprehend basic arguments.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#529

Post by senordevnyc »

datsci_est_2015 wrote: post #353
timschmidt wrote: post #312

Earlier quoted context omitted.

I think there's demonstrably very little difference at all between human and AI outputs, and that's exactly what freaks people out about it. Else they wouldn't be so obsessed with trying to find and define what makes it different. The Thesis of Everything is a Remix is that there is no difference in how any culture is produced. Different models will have a different flavor to their output in the same way as different…

> I think there's demonstrably very little difference at all between human and AI outputs Bold claim, as the internet is awash with counterexamples. In any case, as I think this conversation is trending towards theories of artistic expression, “AI content” will never be truly relatable until it can feel pleasure, pain, and other human urges. The first thing I often think about when I critically assess a piece of art,…

In any case, as I think this conversation is trending towards theories of artistic expression, “AI content” will never be truly relatable until it can feel pleasure, pain, and other human urges. The first thing I often think about when I critically assess a piece of art, like music, is what the artist must have been feeling when they created it, and what prompted them to feel that way.

I recently watched "Come See Me in the Good Light", about the life and death of poet Andrea Gibson. I find their poetry very moving, precisely because it's dripping with human emotion.

Or at least, that's the story I tell myself. The reality is that I perceive it to be written by a human full of emotion. If I were to find out it was AI, I would immediately lose interest, but I think we're already at the point where AI output is indistinguishable from human output in many cases, and if I perceive art to be imbued with human emotion, the actuality of it only matters in terms of how it shapes my perception of it.

I'm not really sure where we'll go with that from here. Maybe art will remain human-created only, and we'll demand some kind of proof of its provenance of being borne of a human mind and a human heart. Or maybe younger generations will really care only about how art makes them feel, not what kind of intelligent entity made it. I really don't know.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#530

Post by LatencyKills »

aspenmartin wrote: post #523
gf000 wrote: post #479

Earlier quoted context omitted.

How confident are you that this knowledge was not part of the training data? Was there no stackoverflow questions/replies with it, no tech forum posts, private knowledge bases, etc? Not trying to diminish its results, just one should always assume that LLMs have a rough memory on pretty much the whole of the internet/human knowledge. Google itself was very impressive back then in how it managed to dig out stuff inter…

Do you think AlphaGo is regurgitating human gameplay? No it’s not: it’s learning an optimal policy based on self play. That is essentially what you’re seeing with agents. People have a very misguided understanding of the training process and the implications of RL in verifiable domains. That’s why coding agents will certainly reach superhuman performance. Straw/steel man depending on what you believe: “But they won’t…

Well said.
Post reply on HN