Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

451–460 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#451

Post by al_borland »

NitpickLawyer wrote: post #167
snemvalts wrote: post #65

Earlier quoted context omitted.

Math and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by p…

> I don't see this getting better. We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve?

[dead]

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#452

Post by jacquesm »

swingboy wrote: post #399
jacquesm wrote: post #306

Earlier quoted context omitted.

> e.g. 167,383 * 426,397 = 71,371,609,051 They may be wrong, but so are you.

You could have just checked the math yourself, you know.

My pocket calculator says the same thing and it doesn't even have training data.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#453

Post by virgildotcodes »

snemvalts wrote: post #76
virgildotcodes wrote: post #56

I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. It's this pervasive belief that underlies so much discussion around what it means to be intelligent. The null hypothesis goes out the window. People constantly make comments like "well it's just trying a bunch of stuff until something works" and it seems that they do not pause for a moment to consider whethe…

The ability to learn and infer without absorbing millions of books and all text on internet really does make us special. And only at 20 watts!

We have a tremendous amount of raw information flowing through our brains 24/7 from before we are born, from the external world through all our senses and from within our minds as it attempts to make sense of that information, make predictions, generally reason about our existence, hallucinate alternative realities, etc. etc.

If you were able to somehow capture all that information in full detail as you've had access to by the age of say 25, it would likely dwarf the amount of information in millions of books by several orders of magnitude.

When you are 25 years old and are presented a strange looking ball and told to throw it into a strange looking basket for the first time. You are relying on an unfathomable amount of information turned into knowledge and countless prior experiments that you've accumulated/exercised to that point relating to the way your body and the world works.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#454

Post by eru »

logicprog wrote: post #437
Yizahi wrote: post #404

Earlier quoted context omitted.

LLMs can generate anything by design. LLMs can't understand what they are generating so it may be true, it may be wrong, it may be novel or it may be known thing. It doesn't discern between them, just looks for the best statistical fit. The core of the issue lies in our human language and our human assumptions. We humans have implicitly assigned phrases "truly novel" and "solving unsolved math problem" a certain mean…

If LLMs can come up with formerly truly novel solutions to things, and you have a verification loop to ensure that they are actual proper solutions, I don't understand why you think they could never come up with solutions to impressive problems, especially considering the thread we are literally on right now? That seems like a pure assertion at this point that they will always be limited to coming up with truly novel…

"Truly novel" is fast becoming a True Scotsman.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#455

Post by tovej »

stavros wrote: post #395
tovej wrote: post #391

Earlier quoted context omitted.

Novel is a tricky word. In this case, the LLM produced a python program that was similar to other programs in its corpus, and this oython program generated examples of hypergraphs that hadn't been seen before. That's a new result, but I don't know about novel. The technique was the same as earlier work in this vein. And it seems like not much computational power was needed at all. (The article mentions that an underg…

I have never seen a human produce a Python program that wasn't similar to other programs they'd seem.

So? I certainly have.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#456

Post by energy123 »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

> 67,383 * 426,397 = 71,371,609,051 ... You need to say why it can do some novel tasks but could never do others.

Model interpretability gives us the answers. The reason LLMs can (almost) do new multiplication tasks is because it saw many multiplication problems in its training data, and it was cheaper to learn the compressed/abstract multiplication strategies and encode them as circuits in the network, rather than memorize the times tables up to some large N. This gives it the ability to approximate multiplication problems it hasn't seen before.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#457

Post by datsci_est_2015 »

stavros wrote: post #444
datsci_est_2015 wrote: post #441

Earlier quoted context omitted.

So why then do we stop training LLMs and keep them stored at a specific state? Is it perhaps because the results become terrible and LLMs have a delicate optimal state for general use? This sounds like an even worse case for a model of intelligence.

Nope, it's not that, but it's nice of you to offer a straw man. Makes the argument flow better.

Not entirely a straw man. What is the purpose of storing and retrieving LLMs at a fixed state if not to guarantee a specific performance? Wouldn’t a strong model of intelligence be capable of, to extend your analogy, running without having its hippocampus lobotomized?

Given the precariousness of managing LLM context windows, I don’t think it’s particularly unfair to assume that LLMs that learn without limit become very unstable.

To steelman, if it’s possible, it may be prohibitively expensive. But somehow I doubt it’s possible.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#458

Post by stavros »

datsci_est_2015 wrote: post #457
stavros wrote: post #444

Earlier quoted context omitted.

Nope, it's not that, but it's nice of you to offer a straw man. Makes the argument flow better.

Not entirely a straw man. What is the purpose of storing and retrieving LLMs at a fixed state if not to guarantee a specific performance? Wouldn’t a strong model of intelligence be capable of, to extend your analogy, running without having its hippocampus lobotomized? Given the precariousness of managing LLM context windows, I don’t think it’s particularly unfair to assume that LLMs that learn without limit become ve…

It is, indeed, prohibitively expensive. But it's not impossible. The proof is in the fact that you can fine-tune LLMs.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#459

Post by charcircuit »

qsera wrote: post #324
charcircuit wrote: post #309

Earlier quoted context omitted.

I think they would be hard to find due to how many posts exists along with how things aren't as funny the second time around.

funny things are funny the n-th time around. Or may be it was just not funny and just something new for you..

We have different senses of humor.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#460

Post by datsci_est_2015 »

hackinthebochs wrote: post #443
datsci_est_2015 wrote: post #440

Earlier quoted context omitted.

> And so declaring what LLMs can't do is wildly premature. The opposite is true as well. Emergent complexity isn’t limitless. Just like early physicists tried to explain the emergent complexity of the universe through experimentation and theory, so should we try to explain the emergent complexity of LLMs through experimentation and theory. Specifically not pseudoscience, though.

Sure, that's true as well. But I don't see this as a substantive response given that the only people making unsupported claims in this thread are those trying to deflate LLM capabilities.

So, to review this thread

  - OP asked for someone to make a logical argument for the separation of “training” from “model”
  - I made the argument
  - You cherry picked an argument against my specific example and made an appeal to emergent complexity
  - I pointed out that emergent complexity isn’t limitless
  - “the only people making unsupported claims in this thread are those trying to deflate LLM capabilities”
Post reply on HN