Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amateur armed with ChatGPT solves an Erdős problem

scientificamerican.com

Post reply on HN
Print view View original

411–420 of 607 posts

Re: Amateur armed with ChatGPT solves an Erdős problem

#411

Post by WarmWash »

zeven7 wrote: post #345
petra wrote: post #329

Earlier quoted context omitted.

I don't haven ChatGPT but Gemini and Claude. But how do you make a language model think for 80 minutes ???

I have Gemini and ChatGPT and keep them on the highest thinking settings. ChatGPT will regularly think 40-60 minutes on the same problem that Gemini will think 10-15 minutes on. The quality of ChatGpt’s response is usually a little higher but not that much higher. My takeaway is Gemini is better at thinking faster, maybe has better more dedicated hardware behind it, and I use Gemini if I want a faster answer but Chat…

I have the same experience, where Gemini thinks dramatically less than ChatGPT (or Claude), while achieving 90%-95% of the answer on it's first go. I'm surprised this isn't talked about more, because the difference is stark, usually around a factor of 5. This shows up in benchmarks too, where Gemini consistently uses many fewer tokens per solve.

So while ChatGPT produces a correct and/or thorough result after 10 minutes, Gemini got most of the way there in 2 minutes. The downside being you need to prompt again to get to the same level as ChatGPT, but you also can get ~5 prompts in the same amount of time.

I have claude to, but I use it the least because it limits so quickly. However its thinking time seems to be on par with ChatGPT

Re: Amateur armed with ChatGPT solves an Erdős problem

#412

Post by nkrisc »

etherealG wrote: post #405
bertil wrote: post #336

Earlier quoted context omitted.

> the AI says things like “Interesting!” My experience of those utterance is that it’s purely phatic mimicry: they lack genuine intuitive surprise, it’s just marking a very odd shift in direction. The problem isn’t the lack of path, is that the rhetorical follow-up to those leaps are usually relevant results, so they stream-of-token ends up rapidly over-playing its own conviction. That’s why it’s necessary (and often…

And what I find fascinating is I see similar mimicking by my 5 year old. Perhaps we shouldn’t be so quick to call this a lack of being genuine. Sometimes emotions are learned in humans but we wouldn’t call them fake. I don’t want to declare machines to have emotion outright, but to call mimicry evidence of falsehood is also itself false.

Mimicry is how kids learn the expected reactions to particular emotions. A kid mimicking your surprise doesn’t mean they are surprised (as surprise requires an existing expectation of an outcome they may not have the experience for), but when they do feel genuine surprise, they’ll know how to express it.

Re: Amateur armed with ChatGPT solves an Erdős problem

#413

Post by vjk800 »

adamgordonbell wrote: post #3

Here is the chat: don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative proofs given a "number theory and primitive sets" math problem. Provide a full unconditional proof or disproof of the problem. {{problem}} REMEMBER - this unconditional argument may require non-trivial, creative and novel elements. Then "Thought for 80m 17s" https://chatgpt.com/share/69dd1c83-b164…

I gave the same prompt to Gemini pro. It thought for maybe 3-5 minutes and gave the wrong answer (it claims the statement is not true) with some arguments that I can't understand well enough to disprove.

Re: Amateur armed with ChatGPT solves an Erdős problem

#414

Post by azan_ »

yfee wrote: post #315
azan_ wrote: post #253

Earlier quoted context omitted.

> Capitalism already is a poor allocator of human effort, resources, and energy, why lock in on this specifically? It's absolutely best allocator of human effort there is. It has some problems but compared to alternatives it's almost perfect.

No it is the best of what we know. There’s something else out there that nobody has the imagination to personally figure it out and get alignment toward it. It can also be true that capitalism is transitory to get to a place where much of the capital one needs is invented.

Well of course the discussion is only about systems that actually exist, not ones that not only not exist, but also can't be imagined by anyone.

Re: Amateur armed with ChatGPT solves an Erdős problem

#415

Post by WarmWash »

jgalt212 wrote: post #360
adamgordonbell wrote: post #3

Here is the chat: don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative proofs given a "number theory and primitive sets" math problem. Provide a full unconditional proof or disproof of the problem. {{problem}} REMEMBER - this unconditional argument may require non-trivial, creative and novel elements. Then "Thought for 80m 17s" https://chatgpt.com/share/69dd1c83-b164…

> "Thought for 80m 17s" Is there any good rule of thumb for how many kWh of electricity this is?

Many orders of magnitude less than the energy needed to sustain a human while they work through the problem.

Re: Amateur armed with ChatGPT solves an Erdős problem

#416

Post by WarmWash »

omcnoe wrote: post #296
sigmoid10 wrote: post #219

Earlier quoted context omitted.

>one wonders if stoking the model to be unconventional is part of the success I've long suspected that a lot of these model's real capabilities are still locked behind certain prompts, despite the big labs spending tons of effort on making default responses to simple prompts better. Even really dumb shit like "Answer this: ..." vs "Question: ..." vs "... you'll be judged by " that should have zero impact in an ideal…

Model output reflects on your input, and the effect is self reinforcing over the course of a whole conversation. Color you add around a problem influences the model behavior. A "dumber"/vague framing will get a less insightful solution, or possibly no solution at all. I don't even necessarily think this is a critical flaw - in general it's just the model tuning it's responses to your style of prompt. People utilize L…

Keeping a pristine context is so important that I used two separate conversations whenever doing something meaningful. One is the main task executor, and the other is for me to bounce random problems, thoughts, and ideas off of while doing everything to keep a pristine context in the executor instance.

It's sort of an agentic loop where I am one of the agents

Re: Amateur armed with ChatGPT solves an Erdős problem

#417

Post by sweezyjeezy »

code51 wrote: post #368

Why on earth is nobody here talking about the sudden jump to use von Mangoldt function? The reasoning trace never types Λ, never types "von Mangoldt", and never invokes ∑_{q|n} Λ(q) = log n. There is a clear discontinuity at play. I remember an article on this, maybe a comment by Terence Tao himself, seen here, but cannot find it.

I think that the thought trace is definitely incomplete - you can see cases where it is like and "let's calculate the integral:[no integral calculated]". The train of thought it's on towards the end of the trace looks like an entirely different approach than what it ends up returning, so I think we are just not seeing the part where it hits on the right approach (sadly).

Re: Amateur armed with ChatGPT solves an Erdős problem

#418

Post by threethirtytwo »

simianwords wrote: post #242
missingdays wrote: post #239

Earlier quoted context omitted.

> ChatGPT equalizes intelligence Citation needed

how can you ask this question with on a post titled "Amateur armed with ChatGPT solves an Erdős problem"???? are you looking for some randomised control trial? omg

God, do people not read my posts? I wrote this: "It also exposes their ACTUAL intelligence which is to say most of HN is not too smart."

These types of people need citations for the time of day. They don't know how to debate or discuss in abstract terms. Reality freezes over if no scientific papers exist on the topic.

Re: Amateur armed with ChatGPT solves an Erdős problem

#419

Post by gverrilla »

ndriscoll wrote: post #355
chvid wrote: post #273

Earlier quoted context omitted.

I am curious if there is a “harness” for maths out there (like the system prompt and tool collection in Claude code but for maths instead of coding)? Asking the llm to structure its response in plan and implementation, allowing it to call tools like python, sage, lean etc.

Why wouldn't you just use coding agents and ensure you have e.g. Lean and Mathlib in the environment?

the system prompt could be narrower, for instance. there's no reason for such a harness to know about React stuff, for instance.

Re: Amateur armed with ChatGPT solves an Erdős problem

#420

Post by holoduke »

hmontazeri wrote: post #380
bertil wrote: post #336

Earlier quoted context omitted.

> the AI says things like “Interesting!” My experience of those utterance is that it’s purely phatic mimicry: they lack genuine intuitive surprise, it’s just marking a very odd shift in direction. The problem isn’t the lack of path, is that the rhetorical follow-up to those leaps are usually relevant results, so they stream-of-token ends up rapidly over-playing its own conviction. That’s why it’s necessary (and often…

The new Opus 4.7 thinks quite often with: Hmmmm… Haha anyone else seen this?

Indeed. I think it's the client. Not the model
Post reply on HN