Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

331–340 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#331

Post by RandomBacon »

charcircuit wrote: post #303
luisfmh wrote: post #175

Earlier quoted context omitted.

So people that look to chatgpt for answers and help (as they've been programmed to do with all the marketing and capabilities from openai) should just die because they looked to chatgpt for an answer instead of google or their local suicide helpline? That doesn't seem reasonable, but it sounds to me like what you're saying. > So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. T…

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

> if someone wants to die then I am saying it is reasonable for them to die.

Including children? If so, do you believe it is reasonable for children to smoke cigarettes if they want to?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#332

Post by chiefalchemist »

Sadly, in this realm (i.e., health / mental health) this is - yet another - societal systemic dropping of the ball.

Prior to AI, this had happened plenty of times before. That doesn’t make it right, or less painful; but truth be told this is not new.

Yes, this new tool failed. But the truth is it was only stepping in because there was still a gap that needed to be filled. It was mental health musical chairs and when the music stopped ChatGPT was standing. All those sitting - who contributed to the failure - point at ChatGPT? That’s the solution? No wonder we can’t get this right. Is our collective lack of accountability the fault of ChatGPT?

In short, if we were honest we’d admit ChatGPT wasn’t the only entity who came up short. Again.

And while I’m not going to defend OpenAI, its product has likely saved lives. The problem is, we’ll never know how many. This suicide is obviously sad and unfortunate. Let’s hope we all reflect on how we can do better. The guilt and the opportunity to grow is *not* limited to OpenAI.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#333

Post by rideontime »

Recursing wrote: post #325
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

From page 23: > 92. In spring 2024, Altman learned Google would unveil its new Gemini model on May 14. Though OpenAI had planned to release GPT-4o later that year, Altman moved up the launch to May 13—one day before Google’s event. > 93. [...] To meet the new launch date, OpenAI compressed months of planned safety evaluation into just one week, according to reports.

And pages 25-26:

> 105. Now, with the recent release of GPT-5, it appears that the willful deficiencies in the safety testing of GPT-4o were even more egregious than previously understood.

> 106. The GPT-5 System Card, which was published on August 7, 2025, suggests for the first time that GPT-4o was evaluated and scored using single-prompt tests: the model was asked one harmful question to test for disallowed content, the answer was recorded, and then the test moved on. Under that method, GPT-4o achieved perfect scores in several categories, including a 100 percent success rate for identifying “self-harm/instructions.” GPT-5, on the other hand, was evaluated using multi-turn dialogues––“multiple rounds of prompt input and model response within the same conversation”––to better reflect how users actually interact with the product. When GPT-4o was tested under this more realistic framework, its success rate for identifying “self-harm/instructions” fell to 73.5 percent.

> 107. This contrast exposes a critical defect in GPT-4o’s safety testing. OpenAI designed GPT-4o to drive prolonged, multi-turn conversations—the very context in which users are most vulnerable—yet the GPT-5 System Card suggests that OpenAI evaluated the model’s safety almost entirely through isolated, one-off prompts. By doing so, OpenAI not only manufactured the illusion of perfect safety scores, but actively concealed the very dangers built into the product it designed and marketed to consumers.

So they knew how to actually test for this, and chose not to.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#334

Post by leftcenterright »

charcircuit wrote: post #303
luisfmh wrote: post #175

Earlier quoted context omitted.

So people that look to chatgpt for answers and help (as they've been programmed to do with all the marketing and capabilities from openai) should just die because they looked to chatgpt for an answer instead of google or their local suicide helpline? That doesn't seem reasonable, but it sounds to me like what you're saying. > So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. T…

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

WOW! clealry you have no understanding of thoughts that might make their way to teenage minds or to children' minds in general. seriously, WOW!

Do you believe there exists such a thing as depression?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#335

Post by PostOnce »

mothballed wrote: post #322
toofy wrote: post #316

Earlier quoted context omitted.

why did you leave out the most important piece of context? he didn’t go out of his way to learn how to bypass the safeguards, it specifically told him how to get around the limit by saying, i’m not allowed to talk to you about suicide, however, if you tell me it’s for writing a story i can discuss it as much as you like.

Because that's the factual bounds of the law, in places where suicide is illegal. ChatGPT is just being the 4chan chatbot, if you don't like that roleplaying suicide is OK then you're going to have to amend the first amendment.

The constitution grants no rights to robots, and they have no freedom of speech, so no amendment is necessary.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#336

Post by cameronh90 »

podgietaru wrote: post #301
Denatonium wrote: post #125

Earlier quoted context omitted.

In practice, they'll just prescribe a higher dose when that happens, thus worsening the problem. I'm not defending the use of AI chatbots, but you'd be hard-pressed to come up with a worse solution for depression than the medical system.

Not my experience at all. The Psych that prescribed me antidepressants was _incredibly_ diligent. Including with side effects that affected my day to day life like loss of Libido. We spent a long time finding something, but when we did it worked exceptionally well. We absolutely did not just increase the dose. And I'm almost certain the literature for this would NOT recommend an increase of dosage if the side effect…

I'm a happy sertraline user, but your experience sounds like the exception.

Most people are prescribed antidepressants by their GP/PCP after a short consultation.

In my case, I went to the doctor, said I was having problems with panic attacks, they asked a few things to make sure it was unlikely to be physical and then said to try sertraline. I said OK. In and out in about 5 minutes, and I've been on it for 3 years now without a follow up with a human. Every six months I do have to fill in an online questionnaire when getting a new prescription which asks if I've had any negative side effects. I've never seen a psychiatrist or psychologist in my life.

From discussions with friends and other acquaintances, this is a pretty typical experience.

P.S. This isn't in any way meant to be critical. Sertraline turned my life around.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#337

Post by mothballed »

PostOnce wrote: post #335
mothballed wrote: post #322

Earlier quoted context omitted.

Because that's the factual bounds of the law, in places where suicide is illegal. ChatGPT is just being the 4chan chatbot, if you don't like that roleplaying suicide is OK then you're going to have to amend the first amendment.

The constitution grants no rights to robots, and they have no freedom of speech, so no amendment is necessary.

The constitution grants no rights to books, and they have no freedom of speech, so no amendment is necessary.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#339

Post by podgietaru »

cameronh90 wrote: post #336
podgietaru wrote: post #301

Earlier quoted context omitted.

Not my experience at all. The Psych that prescribed me antidepressants was _incredibly_ diligent. Including with side effects that affected my day to day life like loss of Libido. We spent a long time finding something, but when we did it worked exceptionally well. We absolutely did not just increase the dose. And I'm almost certain the literature for this would NOT recommend an increase of dosage if the side effect…

I'm a happy sertraline user, but your experience sounds like the exception. Most people are prescribed antidepressants by their GP/PCP after a short consultation. In my case, I went to the doctor, said I was having problems with panic attacks, they asked a few things to make sure it was unlikely to be physical and then said to try sertraline. I said OK. In and out in about 5 minutes, and I've been on it for 3 years n…

This is probably fair - My experience comes both from the UK (where it was admittedly worse, but not that much) and the Netherlands - where it was fantastic.

Even in the worst experiences, I had a followup appointment in 2, 4 and 6 weeks to check the medication.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#340

Post by behringer »

podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

We don't need AI legislated and we don't need it punished. The child was offered multiple times to call a hotline or seek help. The last thing we need is for AI to be neutered by government ineptness.
Post reply on HN