Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amateur armed with ChatGPT solves an Erdős problem

scientificamerican.com

Post reply on HN
Print view View original

471–480 of 607 posts

Re: Amateur armed with ChatGPT solves an Erdős problem

#471

Post by [deleted] »

tptacek wrote: post #452
0xBA5ED wrote: post #448

Earlier quoted context omitted.

Consider that you don't want to hear "statistical generation" because it reminds you of the unchangeable nature of the underlying technology and its ultimate limitations that all the money and data centers in the world will never solve. Despite how amazing and useful they are, they are not intelligent agents. Even in this very thread, someone mentioned they thought the thing was capable of feeling an emotion. Was tha…

I responded to your point empirically, with problems not conventionally understood to be solvable with "text generation", and your response was in effect that I must be wrong because I'm afraid you might be right. Not an especially strong debate move. Can you refute the argument I made, or do you just want to claim LLMs are drinking all our water?

[deleted]

Re: Amateur armed with ChatGPT solves an Erdős problem

#472

Post by culi »

lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

DeepSeek also seems capable of solving it. In under 20 minutes

https://chat.deepseek.com/share/nyuz0vvy2unfbb97fv

I guess we should test across other LLMs too

Re: Amateur armed with ChatGPT solves an Erdős problem

#473

Post by 0xBA5ED »

tptacek wrote: post #465
0xBA5ED wrote: post #456

Earlier quoted context omitted.

Well, I don't believe the LLM solved those problems. I believe the user did. The LLM aggregated large amounts of information statistically, then the user read that and realized there was something to it and fixed it. Those accounts don't mention the 1000 other prompts that technical user did that yielded garbage results and the user was intelligent enough to disregard those.

No, that's false, in every example I gave. But I appreciate you making clearer that I correctly ascertained your original claim, that you believe they literally are just random text generators, and that people are simply cherry picking the rare meaningful text out of them. That's what I thought you meant by "statistical text generator", and is why I was moved to comment.

1) I never said random 2) I never said cherry picking RARE meaningful text 3) It is not false in every example you gave just because you say that it is 4) If I didn't know better, I might think you're confused about what statistical means (hint: it's not random)

Re: Amateur armed with ChatGPT solves an Erdős problem

#474

Post by culi »

sweezyjeezy wrote: post #417
code51 wrote: post #368

Why on earth is nobody here talking about the sudden jump to use von Mangoldt function? The reasoning trace never types Λ, never types "von Mangoldt", and never invokes ∑_{q|n} Λ(q) = log n. There is a clear discontinuity at play. I remember an article on this, maybe a comment by Terence Tao himself, seen here, but cannot find it.

I think that the thought trace is definitely incomplete - you can see cases where it is like and "let's calculate the integral:[no integral calculated]". The train of thought it's on towards the end of the trace looks like an entirely different approach than what it ends up returning, so I think we are just not seeing the part where it hits on the right approach (sadly).

Does DeepSeek's solution look more traceable?

https://chat.deepseek.com/share/nyuz0vvy2unfbb97fv

Re: Amateur armed with ChatGPT solves an Erdős problem

#475

Post by elil17 »

lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

I understood this to mean that the ChatGPT output was technically correct, just hard to understand.

Re: Amateur armed with ChatGPT solves an Erdős problem

#477

Post by bijowo1676 »

jgalt212 wrote: post #360
adamgordonbell wrote: post #3

Here is the chat: don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative proofs given a "number theory and primitive sets" math problem. Provide a full unconditional proof or disproof of the problem. {{problem}} REMEMBER - this unconditional argument may require non-trivial, creative and novel elements. Then "Thought for 80m 17s" https://chatgpt.com/share/69dd1c83-b164…

> "Thought for 80m 17s" Is there any good rule of thumb for how many kWh of electricity this is?

the electricity was going to be consumed regardless whether you ask chatGPT or not.

It would have been either idle, or serving other users' requests.

so the incremental kWh consumption is zero, since costs are fixed and sunk.

as a rule of thumb you can lookup the power consumption of the latest nVidia chip, multiply by factor of two or three (to account for cpu/storage/cooling/network/infra)

Re: Amateur armed with ChatGPT solves an Erdős problem

#478

Post by tptacek »

0xBA5ED wrote: post #473
tptacek wrote: post #465

Earlier quoted context omitted.

No, that's false, in every example I gave. But I appreciate you making clearer that I correctly ascertained your original claim, that you believe they literally are just random text generators, and that people are simply cherry picking the rare meaningful text out of them. That's what I thought you meant by "statistical text generator", and is why I was moved to comment.

1) I never said random 2) I never said cherry picking RARE meaningful text 3) It is not false in every example you gave just because you say that it is 4) If I didn't know better, I might think you're confused about what statistical means (hint: it's not random)

No, it's false in each example because I'm either a first or secondhand party to it happening (except for the Erdos thing) and I know it's false.

You managed to include in your blanket and conclusory rebuttal "solving undergrad math problems instantaneously". That was one of my examples because (1) it pertains to the subthread, (2) I was talking about it upthread, and (3) I have direct firsthand knowledge.

As I said elsewhere: I've fed thousands of math problems through ChatGPT (starting with 4o and now with 5.5). They've all been randomized. They do not appear in textbooks. They cover all the ground from late high school trig to university calc III. I do this habitually, every time I work an "interesting" problem, to get critiques on my own work. GPT has been flawless, routinely spotting errors or missed opportunities. If I have any complaint, it's that GPT tends to be too much better than I am at any given point, using concepts from later courses to solve simpler problems.

Square that with the claim you're making.

I can do the same thing with vulnerability research (I've been a vuln researcher since 1996 and I use LLMs to find vulnerabilities). But this thread is about math, and it's even easier to show you're wrong in the context of math.

Re: Amateur armed with ChatGPT solves an Erdős problem

#479

Post by ngruhn »

culi wrote: post #472
lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

DeepSeek also seems capable of solving it. In under 20 minutes https://chat.deepseek.com/share/nyuz0vvy2unfbb97fv I guess we should test across other LLMs too

Do you have any idea if this is correct?

Re: Amateur armed with ChatGPT solves an Erdős problem

#480

Post by SpicyLemonZest »

elil17 wrote: post #475
lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

I understood this to mean that the ChatGPT output was technically correct, just hard to understand.

I haven't reviewed it myself, but when a mathematician calls a proof "quite poor" and experts have to "sift through" it, I would understand that to mean that it's technically incorrect. Errors like "This statement isn't correct, but it points towards a weaker statement that is, and the subsequent steps can be rebuilt on top of the weaker statement" are pretty common in output from both LLMs and math students.
Post reply on HN