Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

341–350 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#341

Post by datsci_est_2015 »

sigmoid10 wrote: post #241
thedatamonger wrote: post #225

Earlier quoted context omitted.

what bothers me is not that this issue will certainly disappear now that it has been identified, but that that we have yet to identify the category of these "stupid" bugs ...

We already know exactly what causes these bugs. They are not a fundamental problem of LLMs, they are a problem of tokenizers. The actual model simply doesn't get to see the same text that you see. It can only infer this stuff from related info it was trained on. It's as if someone asked you how many 1s there are in the binary representation of this text. You'd also need to convert it first to think it through, or use…

Okay but, genuinely not an expert on the latest with LLMs, but isn’t tokenization an inherent part of LLM construction? Kind of like support vectors in SVMs, or nodes in neural networks? Once we remove tokenization from the equation, aren’t we no longer talking about LLMs?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#342

Post by KellyCriterion »

jacquesm wrote: post #306
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

> e.g. 167,383 * 426,397 = 71,371,609,051 They may be wrong, but so are you.

No, its correct:

https://www.google.com/search?q=167383+*+426397

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#343

Post by stavros »

benterix wrote: post #325
tornikeo wrote: post #308

Earlier quoted context omitted.

Beliefs are not rooted in facts. Beliefs are a part of you, and people aren't all that happy to say "this LLM is better than me"

I'm very happy to say calculators are far better than me in calculations (to a given precision). I'm happy to admit computers are so much better than me in so many aspects. And I have problem saying LLMs are very helpful tools able to generate output so much better than mine in almost every field of knowledge. Yet, whenever I ask it to do something novel or creative, it falls very short. But humans are ingenious beas…

But the question isn't whether you can get LLMs to do something novel, it's whether anyone can get them to do something novel. Apparently someone can, and the fact that you can't doesn't mean LLMs aren't good for that.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#344

Post by SequoiaHope »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

Most inventions are an interpolation of three existing ideas. These systems are very good at that.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#345

Post by datsci_est_2015 »

otabdeveloper4 wrote: post #95
snemvalts wrote: post #65

Earlier quoted context omitted.

Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…

LLMs can often guess the final answer, but the intermediate proof steps are always total bunk. When doing math you only ever care about the proof, not the answer itself.

What’s funny is that there are total cranks in human form that do the same thing. Lots of unsolicited “proofs” being submitted by “amateur mathematicians” where the content is utter nonsense, but like a monkey with a typewriter, there’s the possibility that they stumble upon an incredible insight.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#346

Post by famouswaffles »

staticassertion wrote: post #328
famouswaffles wrote: post #305

Earlier quoted context omitted.

Yes, a "moderately interesting" Open problem. I can't think of any chores that would take an expert months to complete. I can't think of any chores that I've completed but was then 'unconvinced could work'. Please sit down and think about what you are saying here. Are we still talking about chores ? One of the more strange phenomena with machines getting better and the incessant need (seemingly driven by human except…

Writing a complex parser or certainly a compiler is a 1 - 3 month project, for example. Again, I'm not trying to downplay this, but to frame this accurately. I think an AI being able to build a parser/ compiler is cool too. > One of the more strange phenomena with machines getting better and the incessant need (seemingly driven by human exceptionalism) to downplay each result, is that you just end up belittling human…

>Writing a complex parser or certainly a compiler is a 1 - 3 month project, for example.

1. Estimating time completion of something that has been done multiple times before and an open problem that has not yet been solved is a different matter entirely. 1 to 3 months is an educated guess and more likely than not, an underestimate.

2. I do not think months long complex compilers and parsers are being routinely completed by LLMs as your original comment implied. Regardless, they are different classes of problems.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#347

Post by virgildotcodes »

blackcatsec wrote: post #130
slopinthebag wrote: post #91

Earlier quoted context omitted.

I doubt you can even define intelligence sufficiently to argue this point. Since that's an ongoing debate without a resolution thus far. But you claimed that humans aren't unique. I think it's pretty obvious we are on many dimensions including what you might classify as "intelligence". You don't even necessarily have to believe in a "soul" or something like that, although many people do. The capabilities of a human f…

> I often wonder why tech has so many reductionist, materialist, and quite frankly anti-human, thinkers. I think it comes from a position of arrogance/ego. I'll speak for the US here, since that's what I know the most; but the average 'techie' in general skews towards the higher intelligence numbers than the lower parts. This is a very, very broad stroke, and that's intentional to illustrate my point. Because of this…

It's funny because the fundamental thing I'm speaking out against is the arrogance of human exceptionalism.

This whole debate about what it means to be intelligent or human just seems like we're making the same mistakes we've made over and over.

Earth as the center of the universe, sun as the center of the universe, man as the only animal with consciousness and intellect, the anthropomorphic nature of the majority of the deities in our religions and the anthropocentric purpose of the universe within those religions...

I think this desire to believe that we are special, that the universe in some way does ultimately revolve around us, is seemingly a deep need in our psyche but any material analysis of our universe shows that it is extremely unlikely that we hold that position.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#348

Post by stavros »

datsci_est_2015 wrote: post #335
anal_reactor wrote: post #323

Earlier quoted context omitted.

Arguments like "but AI cannot reliably multiply numbers" fundamentally misunderstand how AI works. AI cannot do basic math not because AI is stupid, but because basic math is an inherently difficult task for otherwise smart AI. Lots of human adults can do complex abstract thinking but when you ask them to count it's "one... two... three... five... wait I got lost".

> fundamentally misunderstand how AI works Who does fundamentally understand how LLMs work? Many claims flying around these days, all backed by some of the largest investments ever collectively made by humans. Lots of money to be lost because of fundamental misunderstandings. Personally, I find that AI influencers conveniently brush away any evidence (like inability to perform basic arithmetic) about how LLMs fundame…

You're not doing yourself a favor when you point out "but they can't do arithmetic!" as if anyone says otherwise. Yes, we all know they can't do arithmetic, and that's just how they work.

I feel like I'm saying "this hammer is so cool, it's made driving nails a breeze" and people go "but it can't screw screws in! Why won't anyone talk about that! Hammers really aren't all they're cracked up to be".

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#349

Post by stavros »

LatencyKills wrote: post #333
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

I've been working on a utility that lets me "see through" app windows on macOS [1] (I was a dev on Apple's Xcode team and have a strong understanding of how to do this efficiently using private APIs). I wondered how Claude Code would approach the problem. I fully expected it to do something most human engineers would do: brute-force with ScreenCaptureKit. It almost instantly figured out that it didn't have to "see th…

What was the solution?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#350

Post by veltas »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

Do we know for a fact that LLMs aren't now configured to pass simple arithmetic like this in a simpler calculator, to add illusion of actual insight?
Post reply on HN