Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoch confirms GPT5.4 Pro solved a frontier math open problem

epoch.ai

Post reply on HN
Print view View original

691–700 of 744 posts

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#691

Post by mikkupikku »

eru wrote: post #599
mikkupikku wrote: post #554

Earlier quoted context omitted.

Agreed. We should be asking what the machines measurably can or can't do. If it can't be measured, then it doesn't matter from an engineering standpoint. Does it have a soul? Can't measure it, so it doesn't matter.

That's a bit too pessimistic. Often times you can productively find some measurable proxy for the thing you care about but can't measure. Turing's test is a famous example, of that. Sometimes you only have a one-sided proxy. Eg I can't tell you whether Claude has a soul, but I'm fairly sure my dishwasher ain't.

When push came to shove, it turns out nobody really cared about the Turing test and immediately found excuses to discount it as soon as machines blew through that goalpost. It's fundamentally theological, but the thing is, it doesn't matter. It has no impact on what the machines can demonstrably do.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#692

Post by thaumasiotes »

scoofy wrote: post #690
thaumasiotes wrote: post #679

Earlier quoted context omitted.

> Math is a perfect field for machine learning to thrive because theoretically, all the information ever needed is tied up in the axioms. Not really; the normal way that math progresses, just like everything else, is that you get some interesting results, and then you develop the theoretical framework. We didn't receive the axioms; we developed them from the results that we use them to prove.

Axioms are, again, by definition, arbitrary. It is effectively irrelevant that we try to develop axioms so that the framework mirror the real world. Everything falls out of the axioms, period. If you want to change the axioms to better reflect some aspect about life, that's all well and good, but everything will still fall out of the new axioms.

A different set of "everything" will fall out of the new axioms. You can enumerate it, but no one will care.

That doesn't make for a perfect field, or even a good field, for machine learning to thrive in; what we care about is finding useful results. Starting with arbitrary axioms is a good way to prevent that from happening.

Compare this discussion from an algebra textbook I've been reading recently:

-----

The possibility of combining two elements of A(S) to get yet another element of A(S) endows A(S) with an algebraic structure. We recall how this was done: If f, g ∈ A(S), then we combine them to form the mapping fg []. We called fg the product of f and g, and verified that fg ∈ A(S), and that this product obeyed certain rules.

From the myriad of possibilities we somehow selected four particular rules that govern the behavior of A(S) relative to this product.

[...]

To justify or motivate why these four specific attributes of A(S) were singled out, in contradistinction to some other set of properties, is not easy to do. In fact, in the history of the subject it took quite some time to recognize that these four properties played the key role. We have the advantage of historical hindsight, and with this hindsight we choose them not only to study A(S), but also as the chief guidelines for abstracting to a much wider context.

-----

It takes work, a lot of work, to determine what axioms you should use. Where do you think the information necessary to make that determination comes from?

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#693

Post by gpderetta »

zeroonetwothree wrote: post #621
gpderetta wrote: post #438

Earlier quoted context omitted.

After dismissing it for a long time, I have come around to the philosophical zombie argument. I do not believe that LLMs are conscious, but I also no longer believe that consciousness is a prerequisite for intelligence. I think at this point it is hard to deny that LLMs do not possess some form of intelligence (although not necessarily human-like). I think P-zombies is a fitting description.

I don't think P-zombies can exist. There must be some perceptible difference between an intelligence w/ consciousness and one without. The only way there wouldn't be a difference is if we are mistaken about the consciousness (either both have it or neither do).

> There must be some perceptible difference between an intelligence w/ consciousness and one without

I think there are differences, and I think we can make good guesses, but I'm not sure we can reliably classify a P-zombie from a normal human from their behaviour with 100% accuracy..

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#694

Post by scoofy »

thaumasiotes wrote: post #692
scoofy wrote: post #690

Earlier quoted context omitted.

Axioms are, again, by definition, arbitrary. It is effectively irrelevant that we try to develop axioms so that the framework mirror the real world. Everything falls out of the axioms, period. If you want to change the axioms to better reflect some aspect about life, that's all well and good, but everything will still fall out of the new axioms.

A different set of "everything" will fall out of the new axioms. You can enumerate it, but no one will care. That doesn't make for a perfect field, or even a good field, for machine learning to thrive in; what we care about is finding useful results. Starting with arbitrary axioms is a good way to prevent that from happening. Compare this discussion from an algebra textbook I've been reading recently: ----- The possi…

I'm talking about a deductive framework based on by definition arbitrary axioms.

You're talking about an inductive framework in which we create deductive frameworks that model the world we live in as best as we can tell. This is very, very hard work. The flipping back and forth between the inductive framework and the deductive framework -- between modeling reality and discovering and testing new aspects -- is the heart of what knowledge is, but again, this is the dance between the two frameworks.

I'm just talking about that deductive framework, that exists, locked in it's by definition arbitrary axioms. It's in that framework that machine learning should thrive, because all of the propositions possible will fall out of the axioms.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#695

Post by jibal »

IshKebab wrote: post #688
jibal wrote: post #682

Earlier quoted context omitted.

The conclusion that LLMs don't reason is not a consequence of them not being able to do arithmetic, so your argument isn't valid. Also, see https://news.ycombinator.com/newsguidelines.html "Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive. When disagreeing, please reply to the argument inste…

Plenty of humans can't do arithmetic. Can they also not reason. Reasoning isn't a binary switch. It's a multidimensional continuum. AI can clearly reason to some extent even if it also clearly doesn't reason in the same way that a human would.

> Plenty of humans can't do arithmetic. Can they also not reason.

I just pointed out that this isn't valid reasoning ... it's a fallacy of denial of the antecedent. No one is arguing that because LLMs can't do arithmetic, therefore they can't reason. After all, zamalek said that he can't quickly multiply large numbers in his head, but he isn't saying that therefore he can't reason.

> Reasoning isn't a binary switch. It's a multidimensional continuum.

Indeed, and a lot of humans are very bad at it, as is clear from the comments I'm responding to.

> AI can clearly reason to some extent

The claim was about LLMs, not AI. This is like if someone said that chihuahuas are little and someone responded by saying that dogs are tall to some extent.

LLMs do not reason ... they do syntactic pattern matching. The appearance of reasoning is because of all the reasoning by humans that is implicit in the training data.

I've had this argument too many times ... it never goes anywhere. So I won't respond again ... over and out.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#696

Post by dirasieb »

digitalPhonix wrote: post #681
CamperBob2 wrote: post #666

Earlier quoted context omitted.

Furthermore, the LLM isn't doing things "in its head" - the headline feature of GPT LLMs is attention across all previous tokens, all of its "thoughts" are on paper LOL, talk about special pleading. Whatever it takes to reshape the argument into one you can win, I guess... LLMs don't reason. Let's see you do that multiplication in your head. Then, when you fail, we'll conclude you don't reason. Sound fair?

I can do it with a scratch pad. And I can also tell you when the calculation exceeds what I can do in my head and when I need a scratch pad. I can also check a long multiplication answer in my head (casting 9s, last digit etc.) and tell if there’s a mistake. The LLMs also have access to a scratch pad. And importantly don’t know when they need to use it (as in, they will sometimes get long multiplication right if you…

> And importantly don’t know when they need to use it

patently false, but hey at least you’re able to see the parallel between you with a scratch pad and an LLM with a python terminal

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#697

Post by dirasieb »

jibal wrote: post #674
luma wrote: post #654

Earlier quoted context omitted.

Modern LLMs, just like everyone reading this, will instead reach for a calculator to perform such tasks. I can't do that in my head either, but a python script can so that's what any tool-using LLM will (and should) do.

LLMs don't use tools. Systems that contain LLMs are programmed to use tools under certain circumstances.

you’re just abstracting it away into this new “systems” definition

when someone says LLMs today they obviously mean software that does more than just text, if you want to be extra pedantic you can even say LLMs by themselves can’t even geenrate text since they are just model files if you don’t add them to a “system” that makes use of that model files, doh

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#698

Post by aurizon »

int_19h wrote: post #649
aurizon wrote: post #615

This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... We have all heard this, pity the poor proof readers who will proof them all in a search for the holy grail = zero errors. In a similar way, LLM's are permutational cross associating engines, matched with sieves to filter out the dross. Less filtering = more dross, AKA slop. It can certainly create enormous masses…

> This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... "Eventually" here is something on the order of a few expected lifespans of the universe. The fact that we're getting meaningful results out of LLMs on a human timescale means that they're doing something very different.

Yes, the space is indeed deep/wide, but LLMs probably cull the herd as they proceed so they eliminate swathes as they go. Smart fuzzing in a way.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#699

Post by TheLNL »

razorbeamz wrote: post #104
TheLNL wrote: post #100

Earlier quoted context omitted.

What are you trying to point out here ? Is there any question you can ask today that is not dependent on some existing knowledge that an AI would have seen ?

The point I'm trying to make is that all LLM output is based on likelihood of one word coming after the next word based on the prompt. That is literally all it's doing. It's not "thinking." It's not "solving." It's simply stringing words together in a way that appears most likely. ChatGPT cannot do math. It can only string together words and numbers in a way that can convince an outsider that it can do math. It's a p…

> ChatGPT cannot do math. It can only string together words and numbers in a way that can convince an outsider that it can do math

What am I as a human doing when I "Do math" ?

1.I am looking at the problem at hand, identifying what I have and what I need to get

2.I am then doing a prediction using my pretrained neural net to find possible courses of action to go in a direction that "feels" right

3.I am using my pretrained neural net to find pairs of values that I can substitute with each other (Think multiplication tables, standard results, etc...)

4.Repeat till I arrive at the answer or give up.

As a simple example, when I try to find 600×74+42 I remember the steps for multiplication. I recall the associated pairs of numbers from my tables and complete the multiplication step by step. I then recall the associated pairs of numbers for addition of single digits and add from left to right.

We need to remember that just because we are fast at doing this and are able to do it subconsciously it doesn't mean that we can natively do math, we just do association of information using the neural networks we have trained.

Re: Epoch confirms GPT5.4 Pro solved a frontier math open problem

#700

Post by fenomas »

staticassertion wrote: post #614
qnleigh wrote: post #603

Earlier quoted context omitted.

Actually here's an even better list of progress on a number of open math problems, with plenty of caveats and exposition: https://github.com/teorth/erdosproblems/wiki/AI-contribution...

This is great observational data but it's an early "step 1", I'd definitely need to see an actual analysis of these cases and likely want to have that analysis involve a review of relevant training data.

What you're asking for is exactly what's in the link you replied about. It collects analysis of each solution (or attempt), and info about whether the AI's solution could be found anywhere in the literature.
Post reply on HN