Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

601–610 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#601

Post by qnleigh »

scoofy wrote: post #600
qnleigh wrote: post #597

Earlier quoted context omitted.

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

I studied philosophy focusing on the analytic school and proto-computer science. LLMs are going to force many people start getting a better understanding about what "Knowledge" and "Truth" are, especially the distinction between deductive and inductive knowledge. Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. In the empirical wor…

Where can I read about how LLMs have changed epistemology? Is there a field of philosophy that tries to define and understand 'intelligence'? That sounds very interesting.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#602

Post by scoofy »

qnleigh wrote: post #601
scoofy wrote: post #600

Earlier quoted context omitted.

I studied philosophy focusing on the analytic school and proto-computer science. LLMs are going to force many people start getting a better understanding about what "Knowledge" and "Truth" are, especially the distinction between deductive and inductive knowledge. Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. In the empirical wor…

Where can I read about how LLMs have changed epistemology? Is there a field of philosophy that tries to define and understand 'intelligence'? That sounds very interesting.

There is already philosophy of mind, but it was pretty young when I was in grad school, which was really at the dawn of deep learning algorithms.

I’d say the two most important topics here are philosophy of language (understanding meaning) and philosophy of science (understanding knowledge).

I’ve already mentioned the language philosophers in an edit above, but in philosophy of science I’d add Popper as extremely important here. The concept of negative knowledge as the foundation of empirical understanding seems entirely lost on people. The Black Swan, by Nassim Taleb is a very good casual read on the subject.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#603

Post by qnleigh »

qnleigh wrote: post #598
staticassertion wrote: post #515

Earlier quoted context omitted.

I think "novel" is ill defined here, perhaps. LLMs do appear to be poor general reasoners[0], and it's unclear if they'll improve here. It would be unintuitive for them to be good at this, given that we know exactly how they're implemented - by looking at text and then building a statistical model to predict the next token. From this, if we wanted to commit to LLMs having generalizable knowledge, we'd have to assume…

> I think "novel" is ill defined here That's exactly my point. When people say "LLMs will never do something novel," they seem to be leaning on some vague, ill-defined notion of novelty. The burden of proof is then to specify what degree of novelty is unattainable and why. As for evidence that they can do novel things, there is plenty: 1. I really did ask Gemini to multiply 167,383 * 426,397 before posting this quest…

Actually here's an even better list of progress on a number of open math problems, with plenty of caveats and exposition:

https://github.com/teorth/erdosproblems/wiki/AI-contribution...

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#604

Post by fenomas »

datsci_est_2015 wrote: post #593
fenomas wrote: post #533

Earlier quoted context omitted.

The post you replied to was: > We went from 2 + 7 = 11 to "solved a frontier math problem" in 3 years, yet people don't think this will improve? All that says is that the speaker thinks models will improve past where they are today. Not that it's a logical certainty (the first thing you jumped on them for), and certainly not anything about "limitless potential for growth" (which nobody even mentioned). With replies l…

> All that says is that the speaker thinks models will improve past where they are today. Not that it's a logical certainty Exceedingly generous interpretation in my opinion. I tend to interpret rhetorical questions of that form as “it’s so obvious that I shouldn’t even have to ask it”.

> generous interpretation

The term of art for that is steelmanning, and HN tries to foster a culture of it. Please check the guidelines link in the footer and ctrl+f "strongest".

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#605

Post by porphyra »

qnleigh wrote: post #597
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

There are ways to go beyond the human-quality data limitation. AI can be trained on better quality than average human data because many problems are easy to verify their solutions. For example, in theory, reinforcement learning with an automatic grader on competitive programming problems can lead to an LLM that is better than humans at it.

It's also possible that there can be emergent capabilities. Perhaps a little obtuse, but you can say that humans are trained on human-quality data too and yet brilliant scientists and creative minds can rise above the rest of us.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#606

Post by muyuu »

qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

It's fear.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#608

Post by [deleted] »

qnleigh wrote: post #597
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

[deleted]

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#609

Post by thangalin »

eru wrote: post #599
mikkupikku wrote: post #554

Earlier quoted context omitted.

Agreed. We should be asking what the machines measurably can or can't do. If it can't be measured, then it doesn't matter from an engineering standpoint. Does it have a soul? Can't measure it, so it doesn't matter.

That's a bit too pessimistic. Often times you can productively find some measurable proxy for the thing you care about but can't measure. Turing's test is a famous example, of that. Sometimes you only have a one-sided proxy. Eg I can't tell you whether Claude has a soul, but I'm fairly sure my dishwasher ain't.

Claude has neither a soul nor a warbleflupper.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#610

Post by dyauspitr »

alberth wrote: post #15

For those, like me, who find the prompt itself of interest … > A full transcript of the original conversation with GPT-5.4 Pro can be found here [0] and GPT-5.4 Pro’s write-up from the end of that transcript can be found here [1]. [0] https://epoch.ai/files/open-problems/gpt-5-4-pro-hypergraph-... [1] https://epoch.ai/files/open-problems/hypergraph-ramsey-gpt-5...

I wonder what was in that solutions file they provided. According to the prompt it’s a solution template but I want to know the contents.

Another thing I want to know is how the user keeps updating the LLM with the token usage. I didn’t know they could process additional context midtask like that.

Post reply on HN