Earlier quoted context omitted.
https://news.ycombinator.com/item?id=47495568
what bothers me is not that this issue will certainly disappear now that it has been identified, but that that we have yet to identify the category of these "stupid" bugs ...
Epoch confirms GPT5.4 Pro solved a frontier math open problem
241–250 of 744 posts
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#242Earlier quoted context omitted.
Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…
> I don't see this getting better. We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve?
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#243I like to imagine that the number of consumed tokens before a solution is found is a proxy for how difficult a problem is, and it looks like Opus 4.6 consumed around 250k tokens. That means that a tricky React refactor I did earlier today at work was about half as hard as an open problem in mathematics! :)
You're kidding, but it could be true? Many areas of mathematics are, first and foremost, incredibly esoteric and inaccessible (even to other mathematicians). For this one, the author stated that there might be 5-10 people who have ever made any effort to solve it. Further, the author believed it's a solvable problem if you're qualified and grind for a bit. In software engineering, if only 5-10 people in the world hav…
It's afterall still a remix machine, it can only interpolate between that which already exists. Which is good for a lot of things, considering everything is a remix, but it can't do truly new tasks.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#244I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
AI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the context. That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.
Standard problem*5 + standard solutions + standard techniques for decomposing hard problems = new hard problem solved
There is so much left in the world that hasn’t had anyone apply this approach purely because no research programme has decides that it’s worth their attention.
If you want to shift the bar for “original” beyond problems that can be abstracted into other problems then you’re expecting AI to do more than human researchers do.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#245Earlier quoted context omitted.
This is like saying chess engines don't actually "play" chess, even though they trounce grandmasters. It's a meaningless distinction, about words (think, reason, ..) that have no firm definitions.
Chess engines are not a comparable thing. Chess is a solved game. There is always a mathematically perfect move.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#246I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
>it really is a new and exciting world... The point is that from now on, there will be nothing really new, nothing really original, nothing really exciting. Just endless stream of re-hashed old stuff that is just okayish.. Like an AI spotify playlist, it will keep you in chains (aka engaged) without actually making you like really happy or good. It would be like living in a virtual world, but without having anything…
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#247Earlier quoted context omitted.
sigh; this argument is the new Chinese Room; easily described, utterly wrong. https://www.youtube.com/watch?v=YEUclZdj_Sc
Next-token-prediction cannot do calculations. That is fundamental. It can produce outputs that resemble calculations. It can prompt an agent to input some numbers into a separate program that will do calculations for it and then return them as a prompt. Neither of these are calculations.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#248Earlier quoted context omitted.
This is like saying chess engines don't actually "play" chess, even though they trounce grandmasters. It's a meaningless distinction, about words (think, reason, ..) that have no firm definitions.
Chess engines are not a comparable thing. Chess is a solved game. There is always a mathematically perfect move.
This is a good example of being confidently misinformed.
The best move is always a result of calculation. And the calculation can always go deeper or run on a stronger engine.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#249Earlier quoted context omitted.
>Meaning however you (reasonably) define intelligence, if you compare humans to any AI system humans are overwhelmingly more capable. Really ? Every Human ? Are you sure ? because I certainly wouldn't ask just any human for the things I use these models for, and I use them for a lot of things. So, to me the idea that all humans are 'overwhelmingly more capable' is blatantly false. >Defining "intelligence" as "solving…
I never said that humans are better than LLM's along every axis. Rather, a reasonable definition of intelligence would necessarily encompass domains that LLM's are either incapable of or inferior to us. Here might be some definitions of intelligence for example: > The aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment. > "...the result…
So all humans are overwhelmingly more intelligent but cannot even manage to be as capable in a significant number of domains ? That's not what overwhelming means.
>I would consider statistical reasoning systems that can simulate aspects of human thought to be a form of brute force.
That is not really what “brute force” means. Pattern learning over a compressed representation of experience is not the same thing as exhaustive search. Calling any statistical method “brute force” just makes the term too vague to be useful.
> what is more important to me is the result of that - how does the process of employing our intelligence look.
But this is exactly where you are smuggling in assumptions. We do not actually understand the internal workings of either the human brain or frontier LLMs at the level needed to make confident claims like this. So a lot of what you are calling “the result” is really just your intuition about what intelligence is supposed to look like.
And I do not think that distinction is as meaningful as you want it to be anyway. Flight is flight. Birds fly and planes fly. A plane is not a “simulacrum of flight” just because it achieves the same end by a different mechanism.
>The transcript lacks the vector embeddings of the model's reasoning. It's literally just a summary from the model - not even that really.
You do not need access to every internal representation to see that the model did not arrive at the answer by brute-forcing all possibilities. The observed behavior is already enough to rule that out.
> Do you realize how much compute it would take to run a full simulation of the human brain on a computer ? The most powerful super computer on the planet could not run this in real time.
>You're so close to getting it lol.
No you don't understand what I'm saying. If we were to be more accurate to the brain in silicon, it would be even less efficient than LLMs, never mind humans. Does that mean how the brain works is wrong ? No it means we are dealing with 2 entirely different substrates and directly comparing efficiencies like that to show one is superior is silly.
Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem
#250I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
>it really is a new and exciting world... The point is that from now on, there will be nothing really new, nothing really original, nothing really exciting. Just endless stream of re-hashed old stuff that is just okayish.. Like an AI spotify playlist, it will keep you in chains (aka engaged) without actually making you like really happy or good. It would be like living in a virtual world, but without having anything…
Is it because the AI is trained with existing data? But, we are also trained with existing data. Do you think that there's something that makes human brain special (other than the hundreds of thousands years of evolution but that's what AI is all trying to emulate)?
This may sound hostile (sorry for my lower than average writing skills), but trust me, I'm really trying to understand.