Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

121–130 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#121

Post by refulgentis »

gpm wrote: post #31
johnfn wrote: post #19

Earlier quoted context omitted.

I am kind of joking, but I actually don't know where the flaw in my logic is. It's like one of those math proofs that 1 + 1 = 3. If I were to hazard a guess, I think that tokens spent thinking through hard math problems probably correspond to harder human thought than tokens spend thinking through React issues. I mean, LLMs have to expend hundreds of tokens to count the number of r's in strawberry. You can't tell me…

Some thoughts. 1. LLMs aren't "efficient", they seem to be as happy to spin in circles describing trivial things repeatedly as they are to spin in circles iterating on complicated things. 2. LLMs aren't "efficient", they use the same amount of compute for each token but sometimes all that compute is making an interesting decision about which token is the next one and sometimes there's really only one follow up to the…

I fear that under those constraints, the only optimal output is “42”

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#122

Post by energy123 »

trick-or-treat wrote: post #108
energy123 wrote: post #68

Earlier quoted context omitted.

Models based on RL are still just remixers as defined above, but their distribution can cover things that are unknown to humans due to being present in the synthetic training data, but not present in the corpus of human awareness. AlphaGo's move 37 is an example. It appears creative and new to outside observers, and it is creative and new, but it's not because the model is figuring out something new on the spot, it's…

> the model is summoning those patterns at inference time. You can make that claim about anything: "The human isn't being creative when they write a novel, they're just summoning patterns at typing time". AlphaGo taught itself that move, then recalled it later. That's the bar for human creativity and you're holding AlphaGo to a higher standard without realizing it.

I can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce.

AlphaGo didn't teach itself that move. The verifier taught AlphaGo that move. AlphaGo then recalled the same features during inference when faced with similar inputs.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#123

Post by catlifeonmars »

Validark wrote: post #33

I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.

Are the only two options AI doubter and AI believer?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#124

Post by razorbeamz »

trick-or-treat wrote: post #120
razorbeamz wrote: post #104

Earlier quoted context omitted.

The point I'm trying to make is that all LLM output is based on likelihood of one word coming after the next word based on the prompt. That is literally all it's doing. It's not "thinking." It's not "solving." It's simply stringing words together in a way that appears most likely. ChatGPT cannot do math. It can only string together words and numbers in a way that can convince an outsider that it can do math. It's a p…

> all LLM output is based on likelihood of one word coming after the next word based on the prompt. Right but it has to reason about what that next word should be. It has to model the problem and then consider ways to approach it.

No, it does not reason anything. LLM "reasoning" is just an illusion.

When an LLM is "reasoning" it's just feeding its own output back into itself and giving it another go.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#125

Post by jamesfinlayson »

otabdeveloper4 wrote: post #95
snemvalts wrote: post #65

Earlier quoted context omitted.

Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…

LLMs can often guess the final answer, but the intermediate proof steps are always total bunk. When doing math you only ever care about the proof, not the answer itself.

Yep, I remember a friend saying they did a maths course at university that had the correct answer given for each question - this was so that if you made some silly arithmetic mistake you could go back and fix it and all the marks were for the steps to actually solve the problem.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#126

Post by trick-or-treat »

energy123 wrote: post #122
trick-or-treat wrote: post #108

Earlier quoted context omitted.

> the model is summoning those patterns at inference time. You can make that claim about anything: "The human isn't being creative when they write a novel, they're just summoning patterns at typing time". AlphaGo taught itself that move, then recalled it later. That's the bar for human creativity and you're holding AlphaGo to a higher standard without realizing it.

I can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce. AlphaGo didn't teach itself…

> The verifier taught AlphaGo that move

Ok so it sounds like you want to give the rules of Go credit for that move, lol.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#127

Post by pastel8739 »

_vertigo wrote: post #118
pastel8739 wrote: post #86

Earlier quoted context omitted.

Sure, that may be. But “creativity” is much harder to define and to prove or disprove. My point is that “remixing” does not prohibit new output.

I don’t think that is a good example. No one is debating whether LLMs can generate completely new sequences of tokens that have never appeared in any training dataset. We are interested not only in novel output, we are also interested in that output being correct, useful, insightful, etc. Copying a sequence from the user’s prompt is not really a good demonstration of that, especially given how autoregression/attentio…

Perhaps I should have quoted the parent:

> That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.

My only claim is that precisely this is incorrect.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#128

Post by XenophileJKO »

razorbeamz wrote: post #114
XenophileJKO wrote: post #110

Earlier quoted context omitted.

So this idea that they replay "text" they saw before is kind of wrong fundamentally. They replay "abstract concepts of varied conceptual levels".

The important point I'm trying to reinforce is that LLMs are not capable of calculation. They can give an answer based on the fact that they have seen lots of calculations and their results, but they cannot actually perform mathematical functions.

That is a pretty bold assertion for a meatball of chemical and electrical potentials to make.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#129

Post by kuerbel »

jack_pp wrote: post #101
snemvalts wrote: post #65

Earlier quoted context omitted.

Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…

Maybe to get a real breakthrough we have to make programming languages / tools better suited for LLM strengths not fuss so much about making it write code we like. What we need is correct code not nice looking code.

Yes yes

Let it write a black box no human understands. Give the means of production away.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#130

Post by blackcatsec »

slopinthebag wrote: post #91
virgildotcodes wrote: post #69

Earlier quoted context omitted.

Every living thing on Earth is unique. Every rock is unique in virtually infinite ways from the next otherwise identical rock. There are also a tremendous number of similarities between all living things and between rocks (and between rocks and living things). Most ways in which things are unique are arguably uninteresting. The default mode, the null hypothesis should be to assume that human intelligence isn't intere…

I doubt you can even define intelligence sufficiently to argue this point. Since that's an ongoing debate without a resolution thus far. But you claimed that humans aren't unique. I think it's pretty obvious we are on many dimensions including what you might classify as "intelligence". You don't even necessarily have to believe in a "soul" or something like that, although many people do. The capabilities of a human f…

> I often wonder why tech has so many reductionist, materialist, and quite frankly anti-human, thinkers.

I think it comes from a position of arrogance/ego. I'll speak for the US here, since that's what I know the most; but the average 'techie' in general skews towards the higher intelligence numbers than the lower parts. This is a very, very broad stroke, and that's intentional to illustrate my point. Because of this, techie culture gains quite a bit of arrogance around it with regards to the masses. And this has been trained into tech culture since childhood. Whether it be adults praising us for being "so smart", or that we "figured out the VCR", or some other random tech problem that literally almost any human being can solve by simply reading the manual.

What I've found, in the vast majority of technical problem solving cases that average people have challenges with, if they just took a few minutes to read a manual they'd be able to solve a lot of it themselves. In short, I don't believe as a very strong techie that I'm "smarter than most", but rather that I've taken the time to dive into a subject area that most other humans do not feel the need nor desire to do so.

There are objectively hard problems in tech to solve, but the amount of people solving THOSE problems in the tech industry are few and far in between. And so the tech industry as a whole has spent the last decade or two spinning circles on increasingly complex systems to continue feeding their own egos about their own intelligence. We're now at a point that rather than solving the puzzle, most techies are creating incrementally complex puzzles to solve because they're bored of the puzzles that are in front of them. "Let me solve that puzzle by making a puzzle solver." "Okay, now let me make a puzzle solver creation tool to create puzzle solvers to solve the puzzle." and so forth and so forth. At the end of the day, you're still just solving a puzzle...

But it's this arrogance that really bothers me in the tech bro culture world. And, more importantly, at least in some tech bro circles, they have realized that their target to gathering an exponential increase in wealth doesn't lie in creating new and novel ways to solve the same puzzles, but to try and tout AI as the greatest puzzle solver creation tool puzzle solver known to man (and let me grift off of it for a little bit).

Post reply on HN