ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
1–10 of 94 posts
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#2[deleted]
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#3I wonder if any other major AIs (Grok, Claude, Gemini) had similar accidents. And if not, then why?
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#4OpenAI keeping 4o available in ChatGPT was, in my opinion, a sad case of audience capture. The outpouring from some subreddit communities showed how many people had been seduced by its sycophancy and had formed proto-social relationships with it.
Their blogpost about the 5.1 personality update a few months ago showed how much of a pull this section of their customer base had. Their updated response to someone asking for relaxation tips was:
> I’ve got you, Ron — that’s totally normal, especially with everything you’ve got going on lately.
How does OpenAI get it so wrong, when Anthropic gets it so right?
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#5OpenAI keeping 4o available in ChatGPT was, in my opinion, a sad case of audience capture. The outpouring from some subreddit communities showed how many people had been seduced by its sycophancy and had formed proto-social relationships with it. Their blogpost about the 5.1 personality update a few months ago showed how much of a pull this section of their customer base had. Their updated response to someone asking…
> How does OpenAI get it so wrong, when Anthropic gets it so right?
Are you saying people aren't having proto-social relationships with Anthorpic's models? Because I don't think that's true, seems people use ChatGPT, Claude, Grok and some other specific services too, although ChatGPT seems the most popular. Maybe that just reflects general LLM usage then?
Also, what is "wrong" here really? I feel like the whole concept is so new that it's hard to say for sure what is best for actual individuals. It seems like we ("humanity") are rushing into it, no doubt, and I guess we'll find out.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#6Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#7He probably asked for it.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#8OpenAI keeping 4o available in ChatGPT was, in my opinion, a sad case of audience capture. The outpouring from some subreddit communities showed how many people had been seduced by its sycophancy and had formed proto-social relationships with it. Their blogpost about the 5.1 personality update a few months ago showed how much of a pull this section of their customer base had. Their updated response to someone asking…
> and had formed proto-social relationships with it.
I think the term you're looking for is "parasocial."
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#9Based on what I've read, this generation of LLMs should be considered remarkably risky for anyone with suicidal ideation to be using alone.
It's not about the ideation, it's that the attention model (and its finite size) causes the suicidal person's discourse to slowly displace any constraints built into the model itself over a long session. Talk to the thing about your feelings of self-worthlessness long enough and, sooner or later, it will start to agree with you. And having a machine tell a suicidal person, using the best technology we've built to be eloquent and reasonable-sounding, that it agrees with them is incredibly dangerous.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#10OpenAI keeping 4o available in ChatGPT was, in my opinion, a sad case of audience capture. The outpouring from some subreddit communities showed how many people had been seduced by its sycophancy and had formed proto-social relationships with it. Their blogpost about the 5.1 personality update a few months ago showed how much of a pull this section of their customer base had. Their updated response to someone asking…
> How does OpenAI get it so wrong, when Anthropic gets it so right?
I think it's because of two different operating theories. Anthropic is making tools to help people and to make money. OpenAI has a religious zealot driving it because they think they're on the cusp of real AGI and these aren't bugs but signals they're close. It's extremely difficulty to keep yourself in check and I think Altman no longer has a firm grasp on what it possible today.
The first principle is that you must not fool yourself, and you are the easiest person to fool. - Richard P. Feynman