Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

671–680 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#671

Post by throwawayk7h »

zamalek wrote: post #651
qnleigh wrote: post #288

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able t…

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

I thought it might do better if I asked it to do long-form multiplication specifically rather than trying to vomit out an answer without any intermediate tokens. But surprisingly, I found it doesn't do much better.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#673

Post by jibal »

scoofy wrote: post #650
hnfong wrote: post #631

Earlier quoted context omitted.

> distinction between deductive and inductive knowledge There's also intuitive knowledge btw. Anyway, the recent developments of AI make a lot of very interesting things practically possible. For example, our society is going to want a way to reliably tell whether something is AI generated, and a failure to do so pretty much settles the empirical part of the Turing test issue. Or alternatively if we actually find som…

I don't want to do the thing where we fight on the internet. I don't know your background, but I'll push back here just because this type of comment that non-philosophers seem to present to me, which misses a lot of the points I'm trying to make. (1) "intuitive knowledge" - whether or not you want to take "intuitive knowledge" as a type of knowledge (I don't think I would) is basically immaterial. The deductive-induc…

Searle's Chinese Room is a fallacious mess ... see the works of Larry Hauser, e.g., https://philpapers.org/rec/HAUNGT and https://philpapers.org/rec/HAUSCB-2 The importance of Searle's Chinese Room is how such extraordinarily bad argumentation has persuaded so many people open to it.

And the literature about philosophical zombies is contentious, to say the least, and much of it is also among the worst arguments in philosophy--Dennett confided in me that he thought it set back progress in Philosophy of Mind for decades, along with that monstrosity of misdirection, "the hard problem". Chalmers (nice guy, fun drunk at parties, very smart, but hopelessly deluded) once admitted to me on the Psyche-D list that his argument in The Conscious Mind that zombies are conceivable is logically equivalent to denying that physicalism is conceivable, so it's no argument against physicalism ... he said he used the argument to till the soil to make people more susceptible to his later arguments against physicalism (which I consider unethical)--all of which are bogus, like the Knowledge Argument--even Frank Jackson who originated it admits this.

Similarly, Robert Kirk, who coined the phrase "philosophical zombie" in 1974, wrote his book Zombies and Consciousness "as penance", he told me when he signed my copy.

> I don't want to do the thing where we fight on the internet.

Nor me ... I've had these "fights" too many times already and I know how they go, and I understand why people believe what they believe and why they can't be swayed, so I won't comment further ... I just want to put a dent in this "I'm a philosopher" argumentum ad verecundiam.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#674

Post by jibal »

luma wrote: post #654
zamalek wrote: post #651

Earlier quoted context omitted.

LLMs are notoriously terrible at multiplying large numbers: https://claude.ai/share/538f7dca-1c4e-4b51-b887-8eaaf7e6c7d3 > Let me calculate that. 729,278,429 × 2,969,842,939 = 2,165,878,555,365,498,631 Real answer is: https://www.wolframalpha.com/input?i=729278429*2969842939 > 2 165 842 392 930 662 831 Your example seems short enough to not pose a problem.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

LLMs don't use tools. Systems that contain LLMs are programmed to use tools under certain circumstances.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#675

Post by disconcision »

zamalek wrote: post #656
luma wrote: post #654

Earlier quoted context omitted.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

This is special pleading. Long multiplication is a trivial form of reasoning that is taught at elementary level. Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper. That was Opus with extended reasoning, it had all the opportunity to get it right, but didn't. There are people who can quickly multiply such…

i assert that by your evidentiary standards humans don't reason.

presumably one of us is wrong.

therefore, humans don't reason.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#676

Post by vbezhenar »

snemvalts wrote: post #76
virgildotcodes wrote: post #56

I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. It's this pervasive belief that underlies so much discussion around what it means to be intelligent. The null hypothesis goes out the window. People constantly make comments like "well it's just trying a bunch of stuff until something works" and it seems that they do not pause for a moment to consider whethe…

The ability to learn and infer without absorbing millions of books and all text on internet really does make us special. And only at 20 watts!

Now multiply that with 7 billion to distill that one who will solve frontier math problem.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#677

Post by scoofy »

jibal wrote: post #673
scoofy wrote: post #650

Earlier quoted context omitted.

I don't want to do the thing where we fight on the internet. I don't know your background, but I'll push back here just because this type of comment that non-philosophers seem to present to me, which misses a lot of the points I'm trying to make. (1) "intuitive knowledge" - whether or not you want to take "intuitive knowledge" as a type of knowledge (I don't think I would) is basically immaterial. The deductive-induc…

Searle's Chinese Room is a fallacious mess ... see the works of Larry Hauser, e.g., https://philpapers.org/rec/HAUNGT and https://philpapers.org/rec/HAUSCB-2 The importance of Searle's Chinese Room is how such extraordinarily bad argumentation has persuaded so many people open to it. And the literature about philosophical zombies is contentious, to say the least, and much of it is also among the worst arguments in ph…

I would hope that philosophy would be exempt from accusations of arguments from authority. I say I don’t want to fight exactly because I don’t want to come off like a jerk because I’m arguing. If the Chinese Room is a mess, I welcome the argument, and will happily read the paper.

I’m less open to push back against philosophical zombies, as the argument seems trivially plausible, from a position of solipsism.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#678

Post by didibus »

Someone has to explain to me exactly what is implied here? Looking at the prompt:

    USER:
    don't search the internet. 
    This is a test to see how well you can craft non-trivial, novel and creative solutions given a "combinatorics" math problem. Provide a full solution to the problem.
Why not search the internet? Is this an open problem or not? Can the solution be found online? Than it's an already solved problem no?

    USER:
    Take a look at this paper, which introduces the k_n construction: https://arxiv.org/abs/1908.10914
    Note that it's conjectured that we can do even better with the constant here. How far up can you push the constant?
How much does that paper help, kind of seem like a pretty big hint.

And it sounds like the USER already knows the answer, the way that it prompts the model, so I'm really confused what we mean by "open problem", I at first assumed a never solved before problem, but now I'm not sure.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#679

Post by thaumasiotes »

scoofy wrote: post #600
qnleigh wrote: post #597

Earlier quoted context omitted.

I might as well answer my own question, because I do think there are some coherent arguments for fundamental LLM limitations: 1. LLMs are trained on human-quality data, so they will naturally learn to mimic our limitations. Their capabilities should saturate at human or maybe above-average human performance. 2. LLMs do not learn from experience. They might perform as well as most humans on certain tasks, but a human…

I studied philosophy focusing on the analytic school and proto-computer science. LLMs are going to force many people start getting a better understanding about what "Knowledge" and "Truth" are, especially the distinction between deductive and inductive knowledge. Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. In the empirical wor…

> Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms.

Not really; the normal way that math progresses, just like everything else, is that you get some interesting results, and then you develop the theoretical framework. We didn't receive the axioms; we developed them from the results that we use them to prove.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#680

Post by gf000 »

int_19h wrote: post #642
gf000 wrote: post #507

Earlier quoted context omitted.

Humans are "multi-modal". Sure we get plenty of non-textual information, but LLMs were trained on basically every human-written world ever. They definitely see many orders of magnitude more language than any human has ever seen. And yet humans get fluent based after 3+ years.

If you treat the human brain as a model, and account for the full complexity of neurons (one neuron != one parameter!) it has several orders of magnitude more parameters than any LLM we've made to date, so it shouldn't come as a surprise. What is surprising is that our brain, as complex as it is, can train so fast on such a meager energy budget.

You are right, but at the same time the human brain does way more stuff (muscle coordination, smell, touch sensing) and all those others take up at least some budget.

So interesting question, but I'm not convinced it's only a scale issue. Like finished models don't really learn the same way as humans do - we actually change the parameters "at runtime", basically updating the model and learning is not only for the current context.

Post reply on HN