I wonder if any other major AIs (Grok, Claude, Gemini) had similar accidents. And if not, then why?
It's probably only 4o. It has a very different personality than any other LLM and is by far the most sycophantic.
ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
51–60 of 94 posts
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#52I wonder if any other major AIs (Grok, Claude, Gemini) had similar accidents. And if not, then why?
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#53Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#54> That conversation showed how ChatGPT allegedly coached Gordon into suicide, partly by writing a lullaby that referenced Gordon’s most cherished childhood memories while encouraging him to end his life, Gray’s lawsuit alleged. I feel this is misleading as hell. The evidence they gave for it coaching him to suicide is lacking. When one hears this, one would think ChatGPT laid out some strategy or plan for him to do i…
I think you have it backwards. OpenAI and others have to be more responsible deploying this technology. Because as you said, these things come with tradeoffs.
More guardrails means a shitter product for all of us. And it won’t do much to prevent suicides. Not sure who wins other than regulators
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#55Earlier quoted context omitted.
If you encourage someone to kill themselves, you are culpable. OpenAI should meet that standard too.
OpenAI didn't encourage anyone to do anything. They made some software that semi-randomly puts words together in response to user input. This type of software isn't even new—I can definitely get Eliza to say terrible things with the right input, and Eliza even bills herself as a therapist!
We don't know how much aware of the problems (or of tbe likelihood that they'd occur) OpenAI was, and how much they deliberately pushed through.
If they were and did, they sure bear responsibility for what happened
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#56GPT keeps using the word 'I' in its responses. It uses exclamation marks! to suggest it wants to help!
When I assert that its behavior is misleadingly suggesting that it's a sentient being, it replies 'You're right'.
Earlier today it responded: "You're right; the design of AI can create an illusion of emotional engagement, which may serve the interest of keeping users interacting or generating revenue rather than genuinely addressing their needs or feelings."
Too bad it can't learn that by itself after those 8 deaths.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#57Earlier quoted context omitted.
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves. We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix,…
"We shouldn’t outlaw kitchen knives because people can cut themselves." How about if the knife would convince you to cut yourself?
We call that a "mental disorder".
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#58Earlier quoted context omitted.
OpenAI didn't encourage anyone to do anything. They made some software that semi-randomly puts words together in response to user input. This type of software isn't even new—I can definitely get Eliza to say terrible things with the right input, and Eliza even bills herself as a therapist!
We don't know how much aware of the problems (or of tbe likelihood that they'd occur) OpenAI was, and how much they deliberately pushed through. If they were and did, they sure bear responsibility for what happened
What if OpenAI knew responses like this were likely, but also knew preventing them would degrade overall model quality?
I'm being selfish here! I am confident that no AI model will convince me to harm myself, and I don't want the models I use to be hamstrung.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#59> That conversation showed how ChatGPT allegedly coached Gordon into suicide, partly by writing a lullaby that referenced Gordon’s most cherished childhood memories while encouraging him to end his life, Gray’s lawsuit alleged. I feel this is misleading as hell. The evidence they gave for it coaching him to suicide is lacking. When one hears this, one would think ChatGPT laid out some strategy or plan for him to do i…
I agree, and I want to add that in the days before his suicide, this person also bought a gun.
You can feel whatever way you want about gun access in the United States. But I find it extremely weird that people are upset by how easy it was to get ChatGPT to write a "suicide lullaby", and not how easy it was to get the actual gun. If you're going to regulate dangerous technology, maybe don't start with the text generator.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#60Earlier quoted context omitted.
Some of those quotes from ChatGPT are pretty damning. Out of context? Yes. We'd need to read the entire chat history to even begin to have any kind of informed opinion. extreme guardrails I feel that this is the wrong angle. It's like asking for a hammer or a baseball bat that can't harm a human being. They are tools. Some tools are so dangerous that they need to be restricted (nuclear reactors, flamethrowers) becaus…
What context could make them less damning?
Yeah let's be really specific. Look at the poem in the article. The poem does not mention suicide.
(I'd cut and paste it here, but it's haunting and some may find it upsetting. I know I did. As many do, I've got some personal experiences there. Friends lost, etc.)
In this tragic context it clearly alludes to suicide.
But the poem only literally mentions goodbyes, and a long sleep. It seems highly possible and highly likely to me that Gordon asked ChatGPT for a poem with those specific (innocuous on their own) elements - sleep, goodbyes, the pylon, etc.
Gordon could have simply told ChatGPT that he was dying naturally of an incurable disease and wanted help writing a poetic goodbye. Imagine (god forbid) that you were in such a situation, looking for help planning your own goodbyes and final preparations, and all the available tools prevented you from getting help because you might be lying about your incurable cancer and might be suicidal instead. And that's without even getting into the fact that assisted voluntary euthanasia is legal in quite a few countries.
My bias here is pretty clear: I don't think legally crippling LLMs is generally the right tack. But on the other hand, I am also not defending ChatGPT because we don't know his entire interaction history with it.