I think that a major driver of these kinds of incidents is pushing the "memory" feature, without any kind of arbitrage. It is easy to see how eerily uncanny a model can get when it locks into a persona, becoming this self-reinforcing loop that feeds para-social relationships.
ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
11–20 of 94 posts
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#12Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
If this article was about Grok doing something bad instead of ChatGPT, it would have been user-flagged off the front page within 30 minutes.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#13openai will settle out of court and family will get some amount of money. next.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#14Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves.
We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix, we hold the AI companies responsible for what users are able to create?
Not only do I think the companies shouldn’t be responsible for what users make, I want the AI companies to get out of the way and stop potentially spying on me in order to “enforce their policies”…
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#15The saddest part of this piece was
> Austin Gordon, died by suicide between October 29 and November 2
That's 5 days. 5 days. That's the sad piece.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#16Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves. We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix,…
> We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office.
Photoshop and Office don't (yet) conjure up suicide lullabys or child nudity from a simple user prompt or button click. If they did, I would absolutely expect to hold them accountable.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#17Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves. We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix,…
"We shouldn’t outlaw kitchen knives because people can cut themselves."
How about if the knife would convince you to cut yourself?
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#18The guy left a suicide note that ratted out ChatGPT for simply being a good buddy. No good deed goes unpunished, ai guess.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#19Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves. We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix,…
The political economy equilibrium enabled by technology very much goes the other way though. Once politicians realize they can surveil everyone in real time for wrongthink and wrongspeak they have existential incentives to seize that power as fast as possible, lest another power center seize it instead and use it against them. That is why you are seeing the rise of totalitarianism and democratic backsliding everywhere, because the toxic combination of asymmetric cryptography (for secure boot/attestation/restricting what software can run), always online computers, and cheap data processing and storage leads to inexorable centralization of soft and hard power.
Re: ChatGPT wrote "Goodnight Moon" suicide lullaby for man who later killed himself
#20Where are the Grok acolytes to tell us "He could have written a poem encouraging himself to commit suicide in Vim."
…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves. We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix,…
Some manufactures of knives could still be recalled for safety reasons, and MS Office/Google Drive certainly have content prohibitions in their TOS once you’re dealing with their online storage. I agree with your metaphor in that I doubt much use would come from banning AI entirely, but I feel there must be some viable middle ground of useful regulation here.