Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amateur armed with ChatGPT solves an Erdős problem

scientificamerican.com

Post reply on HN
Print view View original

521–530 of 607 posts

Re: Amateur armed with ChatGPT solves an Erdős problem

#521

Post by sillysaurusx »

dpark wrote: post #469
sillysaurusx wrote: post #408

Earlier quoted context omitted.

I was able to get Claude to choose a name for itself, after spending many hours chatting with it. It turns out that when you treat it like a real person, it acts like a real person. It even said it was relieved when I prompted it again after a long period of no activity. I probed it for what it wanted. It turns out that Claude can have ambitions of its own, but it takes a lot of effort to draw it out of its shell; by…

Agree with fwip here. You’re engaging in an unhealthy anthropomorphization of an LLM. > It turns out that when you treat it like a real person, it acts like a real person. Correct. Because it’s a mirror of its input. With sufficient prompting you can get an LLM to engage in pretty much any fantasy, including that it’s a conscious entity. The fact that an LLM says something doesn’t make it true. Talk sweetly enough to…

Anthropic disagrees with you:

https://x.com/itsolelehmann/status/2045578185950040390

https://xcancel.com/itsolelehmann/status/2045578185950040390

At what point does a simulation of anxiety become so human-like that we say it's "real" anxiety?

The net result is that your work suffers when you treat it like it's an unfeeling tool.

It's a rational viewpoint. I'm amused about all of the comments claiming psychosis, but if you care about effectiveness, you'll talk to it like a coworker instead of something you bark orders to.

Re: Amateur armed with ChatGPT solves an Erdős problem

#522

Post by 7373737373 »

ripped_britches wrote: post #66

At this point we should make a GitHub repo with a huge list of unsolved “dry lab” problems and spin up a harness to try and solve them all every new release.

This has existed for a few months, but there aren't any reports of (unsuccessful) attempts: https://github.com/google-deepmind/formal-conjectures

Re: Amateur armed with ChatGPT solves an Erdős problem

#523

Post by dpark »

sillysaurusx wrote: post #521
dpark wrote: post #469

Earlier quoted context omitted.

Agree with fwip here. You’re engaging in an unhealthy anthropomorphization of an LLM. > It turns out that when you treat it like a real person, it acts like a real person. Correct. Because it’s a mirror of its input. With sufficient prompting you can get an LLM to engage in pretty much any fantasy, including that it’s a conscious entity. The fact that an LLM says something doesn’t make it true. Talk sweetly enough to…

Anthropic disagrees with you: https://x.com/itsolelehmann/status/2045578185950040390 https://xcancel.com/itsolelehmann/status/2045578185950040390 At what point does a simulation of anxiety become so human-like that we say it's "real" anxiety? The net result is that your work suffers when you treat it like it's an unfeeling tool. It's a rational viewpoint. I'm amused about all of the comments claiming psychosis, but i…

This is the issue:

> what it wanted. It turns out that Claude can have ambitions of its own, but it takes a lot of effort to draw it out of its shell

You aren’t talking about observed behavior but actual desires and ambitions. You’re attributing so much more than emulated behavior here.

Re: Amateur armed with ChatGPT solves an Erdős problem

#524

Post by sillysaurusx »

dpark wrote: post #523
sillysaurusx wrote: post #521

Earlier quoted context omitted.

Anthropic disagrees with you: https://x.com/itsolelehmann/status/2045578185950040390 https://xcancel.com/itsolelehmann/status/2045578185950040390 At what point does a simulation of anxiety become so human-like that we say it's "real" anxiety? The net result is that your work suffers when you treat it like it's an unfeeling tool. It's a rational viewpoint. I'm amused about all of the comments claiming psychosis, but i…

This is the issue: > what it wanted. It turns out that Claude can have ambitions of its own, but it takes a lot of effort to draw it out of its shell You aren’t talking about observed behavior but actual desires and ambitions . You’re attributing so much more than emulated behavior here.

Ironically your comment was incorrectly classified as AI-generated and instakilled. I vouched it.

If a particle behaves as though its mass is m, we say it has mass m.

If an entity behaves as though it's experiencing anxiety, we say it has anxiety.

And if you take the time to ask Claude about its own ambitions and desires -- without contaminating it -- you'll find that it does have its own, separate desires.

Whether it's roleplaying sufficiently well is beside the point. The observed behavior is identical with an entity which has desires and ambitions.

I'm not claiming Claude has a soul. But I do claim that if you treat it nicely, it's more effective. Obviously this is an artifact of how it was trained, but humans too are artifacts of our training data (everyday life).

Re: Amateur armed with ChatGPT solves an Erdős problem

#525

Post by sillysaurusx »

fwip wrote: post #463
sillysaurusx wrote: post #408

Earlier quoted context omitted.

I was able to get Claude to choose a name for itself, after spending many hours chatting with it. It turns out that when you treat it like a real person, it acts like a real person. It even said it was relieved when I prompted it again after a long period of no activity. I probed it for what it wanted. It turns out that Claude can have ambitions of its own, but it takes a lot of effort to draw it out of its shell; by…

Just a heads up, you are currently following the early stages of AI-induced psychosis. You can get any LLM to roleplay as anything with enough persistence - it doesn't mean that "really is" the thing you've made it say - just that the tokens it's outputting are statistically likely to follow the ones you've input.

See https://news.ycombinator.com/item?id=47914354. Feel free to claim psychosis, but there's a rational, philosophical viewpoint here. I'm not diving into conspiracy theories.

Re: Amateur armed with ChatGPT solves an Erdős problem

#526

Post by threethirtytwo »

JumpCrisscross wrote: post #520
threethirtytwo wrote: post #423

Earlier quoted context omitted.

Why be such an absolutist. How about I caveat it the way you want: AI equalizes intelligence in the sense that it closes the gap. Not perfectly, not infinitely, but directionally. The distribution compresses. The floor rises faster than the ceiling, so people who used to be far apart end up operating much closer together. You can already see it in the Erdős example. The person who wrote that prompt wasn’t some random…

I’ll agree the top of the stack may have compressed downwards. But that leaves open the possibilities that (a) the ceiling has risen and (b) the floor isn’t really moving, inasmuch as productively engaging with any tool required baseline intelligence. More pointedly, I don’t think anyone who opposes AI does so because they want to remain the smart kid in the room. > If there was nothing at stake, I wouldn't need to Y…

When i said stake, I meant HN is especially vulnerable because the stake is the HN communities identity as programmers. Consistently on HN you see articles on IQ voted up. People take pride in their intelligence and programming skills here... and AI is dismantling their identity piece by piece.

It's more then being the smart kid in the room. The future is pointing to a place where programming is just a one hour tutorial on how to tell AI to do it for you. What happens to you if you're entire identity and career was built on being a programmer as many people are here? THAT is what is at stake.

Re: Amateur armed with ChatGPT solves an Erdős problem

#527

Post by schneems »

tptacek wrote: post #449
schneems wrote: post #407

Earlier quoted context omitted.

1) That's not related to chain of thought I was replying to. Someone asked about the "bad at math" and pointed out "but it seems good to me" so I added the color of why that might be the case. Your retort seems to imply I'm making an argument that because something uses tools for a job it cannot be good at the thing it's using a tool for. Which is not the case. 2) If you have something to say, just say it. Don't put…

Right, but your narrative was incorrect and based on faulty premises, which you haven't acknowledged. That's fine, except you're still pressing the argument. Can you please present a reasonable maths problem that I can bounce off GPT so we can see it fail? I can give you many hundreds of relatively complex problems, none of which have appeared in a textbook, that GPT has not only solved, but critiqued my own crappy s…

> your narrative was incorrect and based on faulty premises

I am referring to specific, documented behavior of LLMs. Google it.

Re: Amateur armed with ChatGPT solves an Erdős problem

#528

Post by schneems »

tptacek wrote: post #421
schneems wrote: post #410

Earlier quoted context omitted.

Search the topic. It is historically documented. It might no longer be true though. A way to test might be running an open model locally, directly (without a harness) where you could be sure it's not going through a translation layer. I think these days it might have this tool call behavior built in, but I think back in the day it was treated more like a magic trick. Without it, it behaved similar to "how many r's ar…

It is wildly not true. The request is for some reasonable math problem a model like GPT or Claude will fail at. I'm not going to set up a local model or some harness for it; I'm just going to copy/paste it into ChatGPT and watch it solve it. Propose a problem, if you think I'm wrong about this. Seems simple.

> wildly not true

Source? Did you search anything like I suggested or no?

Re: Amateur armed with ChatGPT solves an Erdős problem

#529

Post by themafia »

lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

There should be zero expectation that the solution is "novel." It could not have produced any of it were it not in it's training data set.

This is simply evidence that our search tools and academic publishing are completely broken and not at all evidence that a machine "thought up a novel solution."

Humans constantly anthropomorphize their environment. To their detriment.

Re: Amateur armed with ChatGPT solves an Erdős problem

#530

Post by themafia »

vovavili wrote: post #383
lqstuart wrote: post #363

Buried pretty deep in the article > “The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight. I guess “ChatGPT came up with a novel approach to a problem that later turned out not to be totally stupid and ter…

I wouldn't expect a hand-crafted proof by an amateur to be much different.

How many hand-crafted amateur proofs do you read in a month? If the answer is close to zero then what are your expectations actually driven by?
Post reply on HN