Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

501–510 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#501

Post by skohan »

rsynnott wrote: post #304
nis0s wrote: post #14

Why did developers spread the idea of AI consciousness for LLMs in the first place? The usefulness and capability of an LLM is orthogonal to its capacity to develop consciousness. I think people would use LLMs with more detachment if they didn’t believe there was something like a person in them, but they would still become reliant on them, regardless, like people did on calculators for math.

I mean, see the outcry when OpenAI briefly nuked GPT-4o in ChatGPT; people acted as if OpenAI had killed their friend. This is of course all deeply concerning, but it does seem likely that the personified LLM is a more compelling product, and more likely to encourage dependence/addiction.

I wonder to what extent the 4o rollback was motivated by this exact case

Re: A teen was suicidal. ChatGPT was the friend he confided in

#502

Post by j_timberlake »

MBCook wrote: post #24
sumeno wrote: post #20

This is a clear example of why the people claiming that using a chatbot for therapy is better than no therapy are... I'll be extremely generous and say misguided. This kid wanted his parents to know he was thinking about this and the chatbot talked him out of it.

How many of these cases exist in the other direction? Where AI chatbots have actively harmed people’s mental health, including possible to the point of self destructive behaviors or self harm? A single positive outcome is not enough to judge the technology beneficial, let alone safe.

This is called the "Man bites dog" bias.

The many people who don't commit suicide because an AI confidant helped them out are never ever gonna make the news. Meanwhile the opposite cases are "TODAY'S TOP HEADLINE" and that's what people discuss.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#503

Post by rsynnott »

skohan wrote: post #501
rsynnott wrote: post #304

Earlier quoted context omitted.

I mean, see the outcry when OpenAI briefly nuked GPT-4o in ChatGPT; people acted as if OpenAI had killed their friend. This is of course all deeply concerning, but it does seem likely that the personified LLM is a more compelling product, and more likely to encourage dependence/addiction.

I wonder to what extent the 4o rollback was motivated by this exact case

As in the removal of 4o, or its reinstatement? Like, the model involved here was 4o AFAICS; if it was related to this case you'd expect them to remove it and bury it, not remove it and return it a few days after.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#504

Post by rpdillon »

slg wrote: post #498
rpdillon wrote: post #494

Earlier quoted context omitted.

Yep, an excellent illustration of the point. Two dead, a dozen injured. What other gun violence happened this week that we'll never hear about?

Has there been some other incident of gun violence this week that resulted in 3 deaths and 17 injuries that hasn't been reported? According to Wikipedia[1] this is the largest mass shooting this year in terms of number of injuries. I guess that isn't worth reporting in the media. [1] - https://en.wikipedia.org/wiki/List_of_mass_shootings_in_the_...

You're using media coverage to critique my point that media coverage is overblown. My only point here is that less than 1/10 of 1% of the shooting deaths in the United States over the last 30 years were related to school shootings.

This shooting is definitely newsworthy. I'm mostly concerned about the other ~800 shooting deaths, mostly suicides, that statistically happened this week that we don't discuss. Suicides in the US are on the rise, and murder rates are down.

https://www.pewresearch.org/short-reads/2025/03/05/what-the-...

Re: A teen was suicidal. ChatGPT was the friend he confided in

#505

Post by esalman »

worldsavior wrote: post #264
broker354690 wrote: post #109

Why isn't OpenAI criminally liable for this? Last I checked: -Signals emitted by a machine at the behest of a legal person intended to be read/heard by another legal person are legally classified as 'speech'. -ChatGPT is just a program like Microsoft Word and not a legal person. OpenAI is a legal person, though. -The servers running ChatGPT are owned by OpenAI. -OpenAI willingly did business with this teenager, letti…

You could also blame Wikipedia for providing suicidal methods for historic reasons or other. Whoever roams the internet is at it's own responsibility. Of course OpenAI is at fault here also, but this is a fight that will never end, and without any seriously valid justification. Just like AI is sometimes bad at coding, same for psychology and other areas where you double check AI.

I am parent to a 4yo. I am also fairly well versed in development and usage of AI and LLM.

When I want an LLM to do something but it won't, I know various ways to bypass that.

If my son is using AI, which he probably will when he is close to middle school age anyway, I will take care to teach him how to use AI responsibly. He'll be smart enough to know how to bypass, but I'll do my best to teach him when to bypass and when not to bypass. That is if the current state of the art and also AI legislation etc. holds.

But I'm just one parent, I have an engineering degree, a PhD, coding, mathematical, and analytical skills. I'm a very small minority. The vast majority of parents out there do not know what's going to hit there kids and how, or they will have very skewed idea about it.

OpenAI should have been the one here to guide a child not to bypass AI and use it responsibily. They did not. No matter how anyone twist the facts, that's the reality here and the child died.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#506

Post by habinero »

imtringued wrote: post #477
habinero wrote: post #458

Earlier quoted context omitted.

I honestly don't hate the idea. On a more serious note, of course there's ways to put in guard rails. LLMs behave like they do because of intentional design choices. Nothing about it is innate.

If you take this idea even a little bit further, you'll end up with licenses for being allowed to speak.

I wasn't being entirely serious. Also, we managed to require drivers licenses without also walking licenses.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#507

Post by itvision »

habinero wrote: post #439
itvision wrote: post #384

Earlier quoted context omitted.

If a stupid chatbot reinforces something you hold dear, maybe you need the help of a professional psychiatrist. And the kid never did. But yeah, let's paint ChatGPT responsible. It's always corporations, not whatever shit he had in his life, including and not limited to his genes.

Are you really blaming a child in crisis for not having the ability to get a psychiatrist? We regulate plenty of things for safety in highly effective and practical ways. Seatbelts in cars. Railings on stairs. No lead in paint.

The concept of "guilt" is foreign to me. I hate it with all my heart.

On the other hand, someone might be held responsible for this, and that's it.

"Might" is the key word here. Given what we've learned, it's difficult to pinpoint who might be responsible.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#508

Post by itvision »

podgietaru wrote: post #401
itvision wrote: post #379

Earlier quoted context omitted.

> We are living in such a horrific moment. We need these things to be legislated. Punished. We need to stop treating them as magic. They had the tools to prevent this. They had the tools to stop the conversation. To steer the user into helpful avenues. No, no, no and no. ChatGPT wasn't the source of his desire to end his life, nor was it the means to do it. It was a "person" to talk to, since he had no such real peop…

I clearly didn't claim that. My own depression was multifaceted, stemming from genetics, upbringing, social isolation. And the help I got was also multifaceted and involved identifying all those causes. Does it not give you pause when you write things like "it's just ChatGPT. Your own attempt at ending your life hasn't seemingly taught you anything." Suicidality is multi-faceted. It's why there's recorded records of…

> Suicidality is multi-faceted.

I agree with this, yet most of this thread blames OpenAI *alone*.

By the way, I've attempted to commit suicide multiple times over the last five years. Some people even saw me do it. I've talked to ChatGPT extensively about it, and it has tried its best to dissuade me but it didn't convince me at all.

Let me tell you something else: Half of my friends know about it. Has anyone done anything to stop me? No.

Yet here on YC: "BURN OPENAI WITH FIRE, HE'S JUST A KID!"

Anything can become a tool for self-destruction if misused, including ChatGPT.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#509

Post by spectraldrift »

webdevver wrote: post #465
pulkitsh1234 wrote: post #461

> In spring 2024, Altman learned Google would unveil its new Gemini model on May 14. Though OpenAI had planned to release GPT-4o later that year, Altman moved up the launch to May 13—one day before Google’s event. > The rushed deadline made proper safety testing impossible. GPT-4o was a multimodal model capable of processing text, images, and audio. It required extensive testing to identify safety gaps and vulnerabil…

Losers aren't talked about, they just lose. The pitchfork crowd is going to be out to get the AI innovators, one way or another. There's no amount of 'safety training' that will exonerate them. Gemini got burned, now its OpenAIs turn. So the calculus is very simple: Do the absolute minimum that's required, and ship it. Sam is proving himself very capable, very rational. OpenAI could scarce wish for a more politically…

Yikes. You’ve mistaken sociopathy for strategy. “Do the absolute minimum” only sounds rational if you’ve already decided other people’s lives have no value. The real pitchfork crowd isn’t coming for innovators; they’re coming for people who think dead teenagers are an acceptable cost of beating Google’s press release by a day.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#510

Post by itvision »

hkpack wrote: post #398
itvision wrote: post #386

I'm appalled by the groupthink in this thread. Imagine if a suicidal person found a book that prompted them to kill themselves. Would you sue the author for that? This is exactly what we have here.

That is exactly not what we have here. LLM is a new phenomenon and the rules dealing with it will be different than with books or any other prior media.

This is exactly a very weak argument here if it's an argument at all.
Post reply on HN