Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amateur armed with ChatGPT solves an Erdős problem

scientificamerican.com

Post reply on HN
Print view View original

171–180 of 607 posts

Re: Amateur armed with ChatGPT solves an Erdős problem

#171

Post by baxtr »

tptacek wrote: post #145
emp17344 wrote: post #143

Earlier quoted context omitted.

Well, hang on a second - it sounds like you may actually disagree with the user who created this thread. That user claims that these systems exhibit “real intelligence”, and success on this Erdos problem is proof. You seem to be making the claim that LLMs are statistical text generators, but statistical text generation is good enough to succeed in certain cases. Those are different arguments. What do you actually bel…

I don't have any opinion about "real intelligence" or not. I'm not a P(doom)er, I don't think we're on the bring of ascending as a species. But I'm also allergic to arguments like "they're just statistical text generators", because that truly does not capture what these things do or what their capabilities are.

Just to clarify because I’m not sure I understand:

So you agree that LLMs are in fact statistical text generators but you don’t like people use that fact in arguments about the capabilities of the things?

Re: Amateur armed with ChatGPT solves an Erdős problem

#172

Post by Eufrat »

dinkumthinkum wrote: post #99
Eufrat wrote: post #11

Earlier quoted context omitted.

I think we should at least ask the latter, if it turned out it cost $100,000 to generate this solution, I would question the value of it. Erdős problems are usually pure math curiosities AFAIK. They often have no meaningful practical applications.

No meaningful, practical applications? You realize that sounds incredibly naive in the history of mathematics, right? People thought this way about number theory in general, and many other things that turned out to have quite important practical applications. Your statement is also a bit odd in that researchers are already paid throughout their whole careers to solve such problems. I don't know.

> You realize that sounds incredibly naive in the history of mathematics, right?

This is after the fact justification. You are arguing that because a thing (number theory) showed practical applications we should have dumped a lot more effort into it. There is no basis for this argument whatsoever; it also seems to involve inventing a time machine. Number theory had no practical applications until the development of public-key cryptography, but you cannot make funding decisions based on the future since it’s unknowable.

Once we get something working, sure, you can justify more aggressive investment. This is not to say that we should not invest in pie-in-the-sky ideas. We absolutely should and need to. Moonshot research or even somewhat esoteric research is vital, but the current investment in AI is so far out of the ballpark of rational. There’s an energy of a fait accompli here, except it’s still very plausible this is all unsustainable and the market implodes instead.

Re: Amateur armed with ChatGPT solves an Erdős problem

#173

Post by baxtr »

0xBA5ED wrote: post #94
resident423 wrote: post #87

Earlier quoted context omitted.

Solving open math problems is strong evidence of intelligence so there's not really any need for rationalization? I don't understand why intelligence would require intent or motive? Isn't intent just the behaviour of making a specific thing happen rather than other things?

The LLM did not solve the problem.

Who did then?

Re: Amateur armed with ChatGPT solves an Erdős problem

#174

Post by Tarq0n »

CSMastermind wrote: post #141
LPisGood wrote: post #88

Some Erdős problems are basically trivial using sophisticated techniques that were developed later. I remember one of my professors, a coauthor of Erdős boasted to us after a quiz how proud he was that he was able to assign an Erdős problem that went unsolved for a while as just a quiz problem for his undergrads.

Worth mentioning, though, that people have already tried running all of them through LLMs at this point. So this is proof of the models actually getting stronger (previous generations of LLMs were unable to solve this one).

Not definitively. LLMs are stochastic with respect to input, temperature and the exact prompt. It's possible that the model was already capable of it but never received the exact right conditions to produce this output.

Re: Amateur armed with ChatGPT solves an Erdős problem

#175

Post by imiric »

CSMastermind wrote: post #141
LPisGood wrote: post #88

Some Erdős problems are basically trivial using sophisticated techniques that were developed later. I remember one of my professors, a coauthor of Erdős boasted to us after a quiz how proud he was that he was able to assign an Erdős problem that went unsolved for a while as just a quiz problem for his undergrads.

Worth mentioning, though, that people have already tried running all of them through LLMs at this point. So this is proof of the models actually getting stronger (previous generations of LLMs were unable to solve this one).

> So this is proof of the models actually getting stronger (previous generations of LLMs were unable to solve this one).

No, it's not.

While I don't dispute that new models may perform better at certain tasks, the fact that someone was able to use them to solve a novel problem is not proof of this.

LLM output is nondeterministic. Given the same prompt, the same LLM will generate different output, especially when it involves a large number of output tokens, as in this case. One of those attempts might produce a correct output, but this is not certain, and is difficult if not impossible for a human not expert in the domain to determine this, as shown in this thread.

Re: Amateur armed with ChatGPT solves an Erdős problem

#176

Post by nadermx »

fwipsy wrote: post #169
nadermx wrote: post #158

Earlier quoted context omitted.

I don't see where it doesn't say he is, I feel its implied. Another source, proves me right? https://www.newscientist.com/article/2511954-amateur-mathema... https://archive.is/oQvO4

It's implied by "no advanced mathematics training?" The article you linked (thanks for the unpaywalled link, by the way) describes him only as an amateur mathematician, but describes Barreto as a math student. If they were both math students, I feel it would say so? Or perhaps you're arguing it's implicit in him having solved the problem? If so, you're just assuming your conclusion. "AI didn't prove it by itself; Pri…

I'm saying that it wasn't a random person who had no training in math, still miraculous achievement; just trying to show they still had to study maths to even understand how to present the problem and verify it.

Re: Amateur armed with ChatGPT solves an Erdős problem

#178

Post by famouswaffles »

slashdave wrote: post #165
resident423 wrote: post #8

I wonder if the rationalizations people come up with for why this isn't real intelligence will be as creative as ChatGPTs solution.

Proving a negative is a pretty high bar. You also have the problem of defining "real intelligence", which I suspect you can't.

Intelligence is Intelligence. It's intelligent because it does intelligent things. If someone feels the need to add a 'real' and 'fake' moniker to it so they can exclude the machine and make themselves feel better (or for whatever reason) then they are the one meant to be doing the defining, and to tell us how it can be tested for. If they can't, then there's no reason to pay attention to any of it. It's the equivalent of nonsensical rambling. At the end of the day, the semantic quibbling won't change anything.

Re: Amateur armed with ChatGPT solves an Erdős problem

#179

Post by utopiah »

vjerancrnjak wrote: post #154
cryptoegorophy wrote: post #41

Earlier quoted context omitted.

Mine took 20min. Pro. https://chatgpt.com/share/69ed83b1-3704-8322-bcf2-322aa85d7a... But I wish I was math smart to know if it worked or not.

Ask it to formalize it in Lean.

If they aren't "smart enough" to know if it work they most likely are also unable to verify if the Lean formalization is indeed the one that matches the problem they were trying to solve.

Re: Amateur armed with ChatGPT solves an Erdős problem

#180

Post by _ccwi »

CSMastermind wrote: post #141
LPisGood wrote: post #88

Some Erdős problems are basically trivial using sophisticated techniques that were developed later. I remember one of my professors, a coauthor of Erdős boasted to us after a quiz how proud he was that he was able to assign an Erdős problem that went unsolved for a while as just a quiz problem for his undergrads.

Worth mentioning, though, that people have already tried running all of them through LLMs at this point. So this is proof of the models actually getting stronger (previous generations of LLMs were unable to solve this one).

Minor aside, these models do not return the same answer every time you prompt it. Makes it harder to reason over their effectiveness.
Post reply on HN