Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

81–90 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#81

Post by podgietaru »

DSingularity wrote: post #71
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

Shoot man glad you are still with us.

Thank you. I am glad too, I sought help, and I got better. I think the state of mental health care is abysmal in a lot of places, and so I get the impulse to try to find help where ever you can. It's why this story actually hit me quite hard, especially after reading the case file.

For anyone reading that feels like that today. Resources do exist for those feeling low. Hotlines, self-guided therapies, communities. In the short term, medication really helped me. In the long term, a qualified mental health practitioner, CBT and Psychotherapy. And as trite as it is, things can get better. When I look back at my attempt it is crazy to me to see how far I've come.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#82

Post by threatofrain »

throwawaybob420 wrote: post #34
MBCook wrote: post #24

Earlier quoted context omitted.

How many of these cases exist in the other direction? Where AI chatbots have actively harmed people’s mental health, including possible to the point of self destructive behaviors or self harm? A single positive outcome is not enough to judge the technology beneficial, let alone safe.

idk dude if your technology encourages a teenager to kill itself and prevents him from alerting his parents via a cry for help, I don’t care how “beneficial” it is.

Although I don't believe current technology is ready for talk therapy, I'd say that anti-depressants can also cause suicidal thoughts and feelings. Judging the efficacy of medical technology can't be done on this kind of moral absolutism.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#83

Post by fatbird »

podgietaru wrote: post #69
lvl155 wrote: post #39

Clearly ChatGPT should not be used for this purpose but I will say this industry (counseling) is also deeply flawed. They are also mis-incentivized in many parts of the world. And if ChatGPT is basing its interactions on the same scripted contents these “professionals” use, that’s just not right. I really wish people in AI space stop the nonsense and communicate more clearly what these LLMs are designed to do. They’r…

Counseling is (or should be) heavily regulated, and if a counselor had given advice about the logistics of whether a noose would hold it's weight, they'd probably be prosecuted. They allowed this. They could easily stop conversations about suicide. They have the technology to do that.

Counseling is a very heavily regulated field. They're considered health care professionals, they're subject to malpractice, and they're certified by professional bodies (which is legally required, and insurance coverage is usually dependent upon licencing status).

Re: A teen was suicidal. ChatGPT was the friend he confided in

#84

Post by MBCook »

rideontime wrote: post #49
MBCook wrote: post #35

Earlier quoted context omitted.

I agree. If there was one death for 1 million saves, maybe. Instead, this just came up in my feed: https://arstechnica.com/tech-policy/2025/08/chatgpt-helped-t...

This is the same case that is being discussed, and your comment up-thread does not demonstrate awareness that you are, in fact, agreeing with the parent comment that you replied to. I get the impression that you read only the headline, not the article, and assumed it was a story about someone using ChatGPT for therapy and gaining a positive outcome.

I did! Because I can’t see past the paywall. I can’t even read the first paragraph.

So the headline is the only context I have.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#85

Post by [deleted] »

dkiebd wrote: post #80
slg wrote: post #79

Earlier quoted context omitted.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

We have many tools in this life that can maim and kill people and we keep them anyway because they are very useful for other purposes. It’s best to exercise some personal responsibility, including not allowing a 16 year old child unattended access to the internet.

[deleted]

Re: A teen was suicidal. ChatGPT was the friend he confided in

#86

Post by holbrad »

slg wrote: post #79
dkiebd wrote: post #68

Earlier quoted context omitted.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

>“we just have to accept that these systems will occasionally kill children”

I think this a generally a good mindset to have.

I just see the hyper obsessive "safety" culture corrupting things. We as a society are so so afraid of any risk that we're paralysing ourselves.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#87

Post by slg »

dkiebd wrote: post #80
slg wrote: post #79

Earlier quoted context omitted.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

We have many tools in this life that can maim and kill people and we keep them anyway because they are very useful for other purposes. It’s best to exercise some personal responsibility, including not allowing a 16 year old child unattended access to the internet.

Yeah, that is why we don’t have any regulations on the manufacturing and sale of stuff like guns or drugs. The only thing in the way of a 16 year old having unfettered access is personal responsibility.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#88

Post by AIPedant »

idle_zealot wrote: post #37
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then

a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder

b) OpenAI would be deeply (and deservedly) vulnerable to civil liability

c) state and federal regulators would be on the warpath against OpenAI

Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit changes 2) and 3) - in fact it makes 3) far more urgent.

[1] It is a somewhat ugly constitutional question whether this speech would be protected if it was between two adults, assuming the other adult was not acting as a caregiver. There was an ugly case in Massachusetts involving where a 17-year-old ordered her 18-year-old boyfriend to kill himself and he did so; she was convicted of involuntary manslaughter, and any civil-liberties minded person understands the difficult issues this case raises. These issues are moot if the speech is between an adult and a child, there is a much higher bar.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#89

Post by AIPedant »

threatofrain wrote: post #82
throwawaybob420 wrote: post #34

Earlier quoted context omitted.

idk dude if your technology encourages a teenager to kill itself and prevents him from alerting his parents via a cry for help, I don’t care how “beneficial” it is.

Although I don't believe current technology is ready for talk therapy, I'd say that anti-depressants can also cause suicidal thoughts and feelings. Judging the efficacy of medical technology can't be done on this kind of moral absolutism.

I think it's fine to be "morally absolutist" when it's non-medical technology, developed with zero input from federal regulators, yet being misused and misleadingly marketed for medical purposes.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#90

Post by ruraljuror »

idle_zealot wrote: post #37
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

I agree with your larger point, but I don't understand what you mean the LLM doesn’t do anything? LLMs do do things and they can absolutely have agency (hence all the agents being released by AI companies).

I don’t think this agency absolves companies of any responsibility.

Post reply on HN