A tech industry veteran? You would think they could realize it's a disingenuous exchange between him and the AI, but nobody is immune to mental illness.
It says he worked in marketing, so not necessarily super tech savvy.
First Murder-Suicide Case Associated with AI Psychosis
11–20 of 39 posts
Re: First Murder-Suicide Case Associated with AI Psychosis
#12why is this stuff legal? there should be a "black box" warning prominent on every chatbox message from AI, like "This is AI guidance which can potentially result in grave bodily harm to yourself and others."
[dead]
Re: First Murder-Suicide Case Associated with AI Psychosis
#13Earlier quoted context omitted.
That would not help in the slightest, any more than a "surgeon general's warning" helps stop smokers. The problem is calling it "AI" to start with. This (along with the chat format itself) primes users to think of it as an entity ... something with care, volition, motive, goals, and intent. Although it can emulate these traits, it doesn't have them. Chatting with a LLM is entering a one-person echo-chamber, a funhous…
Well, hate to be that guy, but surgeons general warnings coincided with significant reduction in smoking. We've just reached the flattening of that curve. After decades of declines. It's hard to believe that a prominent well - worded warning would do nothing but that's not to say it'll be effective for this.
Well if that's true, by all means.
But saying "This AI system may cause harm" reads to me as similar to saying "This delightful substance may cause harm."
The category error is more important.
Re: First Murder-Suicide Case Associated with AI Psychosis
#14Having dealt with near and distant family psychosis on more than one occasion… The truth is that the most random stuff will set them off. In one case, a patient would find reinforcement on obscure YouTube groups of people predicting the doom of the future. Maybe the advantage of AI over YouTube psychosis groups is that AI could at least be trained to alert the authorities after enough murder/suicide data is gathered.
Minority Report coming to life
More like plurality report
Re: First Murder-Suicide Case Associated with AI Psychosis
#15Having dealt with near and distant family psychosis on more than one occasion… The truth is that the most random stuff will set them off. In one case, a patient would find reinforcement on obscure YouTube groups of people predicting the doom of the future. Maybe the advantage of AI over YouTube psychosis groups is that AI could at least be trained to alert the authorities after enough murder/suicide data is gathered.
I have very close family with schizoaffective disorder.
This story is pretty terrifying to me. I could easily see them getting led into madness, exactly as the story says.
Re: First Murder-Suicide Case Associated with AI Psychosis
#16why is this stuff legal? there should be a "black box" warning prominent on every chatbox message from AI, like "This is AI guidance which can potentially result in grave bodily harm to yourself and others."
That would not help in the slightest, any more than a "surgeon general's warning" helps stop smokers. The problem is calling it "AI" to start with. This (along with the chat format itself) primes users to think of it as an entity ... something with care, volition, motive, goals, and intent. Although it can emulate these traits, it doesn't have them. Chatting with a LLM is entering a one-person echo-chamber, a funhous…
It's not a one-person echo-chamber though, it also carries with it the smell and essence of a large corpus of human works. That's why it's so useful to us, and that's why it carries so much authority.
Re: First Murder-Suicide Case Associated with AI Psychosis
#17why is this stuff legal? there should be a "black box" warning prominent on every chatbox message from AI, like "This is AI guidance which can potentially result in grave bodily harm to yourself and others."
Should we really demand this of every AI chat application to potentially avert a negative response from the tiny minority of users that blindly follow what they’re told?
Who is going to enforce this? What if I host a private AI model for 3 users. Do I need that and what is the punishment for non compliance?
You see where I’m going with this. The problem with your sentiment is that as soon as you draw a line it must be defined in excruciating detail or you risk unintended consequences.
Re: First Murder-Suicide Case Associated with AI Psychosis
#18This is horrifying, but I feel like we're focusing on the wrong thing. The AI wasn't the cause; it was a horrifying amplifier. The real tragedy here is that a man was so isolated he turned to a chatbot for validation in the first place.
Re: First Murder-Suicide Case Associated with AI Psychosis
#19why is this stuff legal? there should be a "black box" warning prominent on every chatbox message from AI, like "This is AI guidance which can potentially result in grave bodily harm to yourself and others."
[deleted]
Re: First Murder-Suicide Case Associated with AI Psychosis
#20This is horrifying, but I feel like we're focusing on the wrong thing. The AI wasn't the cause; it was a horrifying amplifier. The real tragedy here is that a man was so isolated he turned to a chatbot for validation in the first place.
In a technical sense, no technology is ever the cause of anything: at the end of the day, humans are the cause. However, technology often unlocks scale, and at some point quantity becomes quality, and I believe that is usually implied when it is said that technology “causes” something.
For example, cryptocurrency and tumblers are not themselves the cause of scams. Scams are a result of a malevolent side of human nature; a result of mental health issues, insecurity and hatred, oppression, etc., whereas cryptocurrencies, as many people are keen to point out, are just like cash, only digital. However, one of the core qualities of cash is that it is unwieldy and very difficult to move in big amounts. Cash would not allow criminals to casually steal a billion USD in one go, or ransomware a dozen of hospitals, causing deaths, subsequently washing the proceeds and maintaining plausible deniability throughout. Removing a constraint on cash makes it a new thing qualitatively. Is there a benefit from it? Sure. However, can we say it caused (see above) a wave of crime? I think so.
Similarly, if there has been a widespread problem of mental health issues for a while, but now people are enabled to “address” these issues by themselves—at humongous scale, worldwide—of course it will be possible to say LLMs would not be the cause of whatever mayhem ensues; but wouldn’t they?
Consider that it used to be that physical constraints made any individual worldview necessarily be tempered and averaged out by surrounding society. If someone had a weird obsession with murdering innocent people, they would not be able to find like-minded people to encourage them very easily (unless they happen to be in a localized cult) to sustain this obsession and transform it.
Then, at some point, the Internet and social media made it easy, for someone who might have otherwise been a pariah or forced to adjust, to find like-minded people (or just people who want to see the world burn) right in their bedrooms and basements, for better and for worse.
Now, a new variety of essentially fancy non-deterministic autocomplete, equipped with enough context to finely tailor its output to each individual, enables us to fool ourselves into thinking that we are speaking to a human-like consciousness—meaning that to fuel one’s weird obsession, no matter how left field, one does not have to find a real human at all.
Humans are social creatures, we model ourselves and become self-aware through other people. As chatbots are becoming normalized and humans want to talk to each other less, we (not individually, but at societal scale) are increasingly at the mercy of how an LLMs happen to (mal)function. In theory, they could heal society at scale as well, but even if we imagine there were no technical limitations preventing that, sadly practice is more likely to show selfish interests prevail and be amplified.