21–30 of 39 posts
This is horrifying, but I feel like we're focusing on the wrong thing. The AI wasn't the cause; it was a horrifying amplifier. The real tragedy here is that a man was so isolated he turned to a chatbot for validation in the first place.
Having dealt with near and distant family psychosis on more than one occasion… The truth is that the most random stuff will set them off. In one case, a patient would find reinforcement on obscure YouTube groups of people predicting the doom of the future. Maybe the advantage of AI over YouTube psychosis groups is that AI could at least be trained to alert the authorities after enough murder/suicide data is gathered.
why is this stuff legal? there should be a "black box" warning prominent on every chatbox message from AI, like "This is AI guidance which can potentially result in grave bodily harm to yourself and others."
That would not help in the slightest, any more than a "surgeon general's warning" helps stop smokers. The problem is calling it "AI" to start with. This (along with the chat format itself) primes users to think of it as an entity ... something with care, volition, motive, goals, and intent. Although it can emulate these traits, it doesn't have them. Chatting with a LLM is entering a one-person echo-chamber, a funhous…
Particularly given some documented instances where a user has asked the language model about similar warnings, and the model responded by downplaying the warnings, or telling the user to disregard them.
Earlier quoted context omitted.
That would not help in the slightest, any more than a "surgeon general's warning" helps stop smokers. The problem is calling it "AI" to start with. This (along with the chat format itself) primes users to think of it as an entity ... something with care, volition, motive, goals, and intent. Although it can emulate these traits, it doesn't have them. Chatting with a LLM is entering a one-person echo-chamber, a funhous…
Well, hate to be that guy, but surgeons general warnings coincided with significant reduction in smoking. We've just reached the flattening of that curve. After decades of declines. It's hard to believe that a prominent well - worded warning would do nothing but that's not to say it'll be effective for this.
Printing and postage cost was about £5.8 million. At the time, I thought it was a waste of taxpayers’ money. A letter wouldn’t change anyone’s behaviour, least of all not mine or anyone I knew.
But the economics told a different story. The average cost of treating a Covid patient in intensive care ran to tens of thousands of pounds. The UK Treasury values preventing a single death at around £2 million (their official Value of a Prevented Fatality). That means if the letter nudged just three people into behaviour that prevented their deaths, it would have paid for itself. Three deaths, out of 30 million households.
In reality, the effect was likely far larger. If only 0.1% of households (30,000 families) changed their behaviour even slightly, whether through better handwashing, reduced contact, or staying home when symptomatic, those small actions would multiply during an exponential outbreak. The result could easily be hundreds of lives saved and millions in healthcare costs avoided.
Seen in that light, £5.8 million wasn’t wasteful at all. It was one of the smarteer investments of the pandemic.
What I dismissed as wasteful and pointless, turned out to be a great example of how what appears to be a large upfront cost can deliver returns that massively outweigh the initial outlay.
I changed my view and admitted I was wrong.
This is horrifying, but I feel like we're focusing on the wrong thing. The AI wasn't the cause; it was a horrifying amplifier. The real tragedy here is that a man was so isolated he turned to a chatbot for validation in the first place.
1) The man became severely mentally ill in middle age, and he lived with his mother because he couldn't take care of himself. Describing him as merely "isolated" makes me wonder if you read the article: meeting new friends was not going to help him very much because he was not capable of maintaining those friendships.
2) Saying people turn to chatbots because of isolation is like saying they turn to drugs because of depression. In many cases that's how it started. But people get addicted to chatbots because they are to social interaction what narcotics are to happiness: in the short term you get all of the pleasure without doing any of the work. Human friends insist on give-and-take, chatbots are all give-give-give.
This man didn't talk to chatbots because he was lonely. He did so because he was totally disconnected from reality, and actual human beings don't indulge delusions with endless patience and encouragement the way ChatGPT does. His case is extreme but "people tell me I'm stupid or crazy, ChatGPT says I'm right" is becoming a common theme on social media. It is precisely why LLMs are so addictive and so dangerous.
Someone is mentally ill, and can use AI. Doesn't mean AI is the problem. A mentally ill person can also use a car. Let's ban cars?
This is not Cyberpunk 2077 and "AI psychosis" is trash, just like the article. Someone is mentally ill, and can use AI. Doesn't mean AI is the problem. A mentally ill person can also use a car. Let's ban cars?
Not saying I want AIs to be banned or that the article is good, I'm just argumenting that your analogy could potentially be flawed.
Having dealt with near and distant family psychosis on more than one occasion… The truth is that the most random stuff will set them off. In one case, a patient would find reinforcement on obscure YouTube groups of people predicting the doom of the future. Maybe the advantage of AI over YouTube psychosis groups is that AI could at least be trained to alert the authorities after enough murder/suicide data is gathered.
At some point you have to just live with marginal dangers. There is no technical solution here.
Having dealt with near and distant family psychosis on more than one occasion… The truth is that the most random stuff will set them off. In one case, a patient would find reinforcement on obscure YouTube groups of people predicting the doom of the future. Maybe the advantage of AI over YouTube psychosis groups is that AI could at least be trained to alert the authorities after enough murder/suicide data is gathered.
I don't think AI informing authorities (who would do nothing) is in any way desirable. At some point you have to just live with marginal dangers. There is no technical solution here.