Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

241–250 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#241

Earlier quoted context omitted.

https://news.ycombinator.com/item?id=47495568

what bothers me is not that this issue will certainly disappear now that it has been identified, but that that we have yet to identify the category of these "stupid" bugs ...

We already know exactly what causes these bugs. They are not a fundamental problem of LLMs, they are a problem of tokenizers. The actual model simply doesn't get to see the same text that you see. It can only infer this stuff from related info it was trained on. It's as if someone asked you how many 1s there are in the binary representation of this text. You'd also need to convert it first to think it through, or use some external tool, even though your computer never saw anything else.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#242

Earlier quoted context omitted.

Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…

> I don't see this getting better. We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve?

I tend to disagree that improvement is inherent. Really I'm just expressing an aesthetic preference when I say this, because I don't disagree that a lot of things improve. But it's not a guarantee, and it does take people doing the work and thinking about the same thing every day for years. In many cases there's only one person uniquely positioned to make a discovery, and it's by no means guaranteed to happen. Of course, in many cases there are a whole bunch of people who seem almost equally capable of solving something first, but I think if you say things like "I'm sure they're going to make it better" you're leaving to chance something you yourself could have an impact on. You can participate in pushing the boundaries or even making a small push on something that accelerates someone else's work. You can also donate money to research you are interested in to help pay people who might come up with breakthroughs. Don't assume other people will build the future, you should do it too! (Not saying you DON'T)

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#243
post #13

I like to imagine that the number of consumed tokens before a solution is found is a proxy for how difficult a problem is, and it looks like Opus 4.6 consumed around 250k tokens. That means that a tricky React refactor I did earlier today at work was about half as hard as an open problem in mathematics! :)

You're kidding, but it could be true? Many areas of mathematics are, first and foremost, incredibly esoteric and inaccessible (even to other mathematicians). For this one, the author stated that there might be 5-10 people who have ever made any effort to solve it. Further, the author believed it's a solvable problem if you're qualified and grind for a bit. In software engineering, if only 5-10 people in the world hav…

If only 5-10 people have ever tried to solve something in programming, every LLM will start regurgitating your own decade-old attempt again and again, sometimes even with the exact comments you wrote back then (good to know it trained on my GitHub repos...), but you can spend upwards of 100mio tokens in gemini-cli or claude code and still not make any progress.

It's afterall still a remix machine, it can only interpolate between that which already exists. Which is good for a lot of things, considering everything is a remix, but it can't do truly new tasks.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#244
post #55

I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.

AI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the context. That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.

Turning a hard problem into a series of problems we know how to solve is a huge part of problem solving and absolutely does result in novel research findings all the time.

Standard problem*5 + standard solutions + standard techniques for decomposing hard problems = new hard problem solved

There is so much left in the world that hasn’t had anyone apply this approach purely because no research programme has decides that it’s worth their attention.

If you want to shift the bar for “original” beyond problems that can be abstracted into other problems then you’re expecting AI to do more than human researchers do.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#245

Earlier quoted context omitted.

This is like saying chess engines don't actually "play" chess, even though they trounce grandmasters. It's a meaningless distinction, about words (think, reason, ..) that have no firm definitions.

Chess engines are not a comparable thing. Chess is a solved game. There is always a mathematically perfect move.

Chess is absolutely not a solved game, outside of very limited situations like endgames. Just because a best move exists does not mean we (or even an engine) know what it is

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#246
post #238

I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.

>it really is a new and exciting world... The point is that from now on, there will be nothing really new, nothing really original, nothing really exciting. Just endless stream of re-hashed old stuff that is just okayish.. Like an AI spotify playlist, it will keep you in chains (aka engaged) without actually making you like really happy or good. It would be like living in a virtual world, but without having anything…

I heard this saying recently “The problem with comfort is that it makes you comfortable.”

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#247

Earlier quoted context omitted.

sigh; this argument is the new Chinese Room; easily described, utterly wrong. https://www.youtube.com/watch?v=YEUclZdj_Sc

Next-token-prediction cannot do calculations. That is fundamental. It can produce outputs that resemble calculations. It can prompt an agent to input some numbers into a separate program that will do calculations for it and then return them as a prompt. Neither of these are calculations.

So you don't think 50T parameter neural networks can encode the logic for adding two n-bit integers for reasonably sized integers? That would be pretty sad.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#248

Earlier quoted context omitted.

This is like saying chess engines don't actually "play" chess, even though they trounce grandmasters. It's a meaningless distinction, about words (think, reason, ..) that have no firm definitions.

Chess engines are not a comparable thing. Chess is a solved game. There is always a mathematically perfect move.

> Chess is a solved game. There is always a mathematically perfect move.

This is a good example of being confidently misinformed.

The best move is always a result of calculation. And the calculation can always go deeper or run on a stronger engine.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#249

Earlier quoted context omitted.

>Meaning however you (reasonably) define intelligence, if you compare humans to any AI system humans are overwhelmingly more capable. Really ? Every Human ? Are you sure ? because I certainly wouldn't ask just any human for the things I use these models for, and I use them for a lot of things. So, to me the idea that all humans are 'overwhelmingly more capable' is blatantly false. >Defining "intelligence" as "solving…

I never said that humans are better than LLM's along every axis. Rather, a reasonable definition of intelligence would necessarily encompass domains that LLM's are either incapable of or inferior to us. Here might be some definitions of intelligence for example: > The aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment. > "...the result…

>I never said that humans are better than LLM's along every axis. Rather, a reasonable definition of intelligence would necessarily encompass domains that LLM's are either incapable of or inferior to us.

So all humans are overwhelmingly more intelligent but cannot even manage to be as capable in a significant number of domains ? That's not what overwhelming means.

>I would consider statistical reasoning systems that can simulate aspects of human thought to be a form of brute force.

That is not really what “brute force” means. Pattern learning over a compressed representation of experience is not the same thing as exhaustive search. Calling any statistical method “brute force” just makes the term too vague to be useful.

> what is more important to me is the result of that - how does the process of employing our intelligence look.

But this is exactly where you are smuggling in assumptions. We do not actually understand the internal workings of either the human brain or frontier LLMs at the level needed to make confident claims like this. So a lot of what you are calling “the result” is really just your intuition about what intelligence is supposed to look like.

And I do not think that distinction is as meaningful as you want it to be anyway. Flight is flight. Birds fly and planes fly. A plane is not a “simulacrum of flight” just because it achieves the same end by a different mechanism.

>The transcript lacks the vector embeddings of the model's reasoning. It's literally just a summary from the model - not even that really.

You do not need access to every internal representation to see that the model did not arrive at the answer by brute-forcing all possibilities. The observed behavior is already enough to rule that out.

> Do you realize how much compute it would take to run a full simulation of the human brain on a computer ? The most powerful super computer on the planet could not run this in real time.

>You're so close to getting it lol.

No you don't understand what I'm saying. If we were to be more accurate to the brain in silicon, it would be even less efficient than LLMs, never mind humans. Does that mean how the brain works is wrong ? No it means we are dealing with 2 entirely different substrates and directly comparing efficiencies like that to show one is superior is silly.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#250
post #238

I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.

>it really is a new and exciting world... The point is that from now on, there will be nothing really new, nothing really original, nothing really exciting. Just endless stream of re-hashed old stuff that is just okayish.. Like an AI spotify playlist, it will keep you in chains (aka engaged) without actually making you like really happy or good. It would be like living in a virtual world, but without having anything…

On what do you base your prediction?

Is it because the AI is trained with existing data? But, we are also trained with existing data. Do you think that there's something that makes human brain special (other than the hundreds of thousands years of evolution but that's what AI is all trying to emulate)?

This may sound hostile (sorry for my lower than average writing skills), but trust me, I'm really trying to understand.

Post reply on HN