A teen was suicidal. ChatGPT was the friend he confided in • Hacker News Board

Re: A teen was suicidal. ChatGPT was the friend he confided in

#421

Post by scotty79 » Wed, Aug 27, 2025, 1:09 PM UTC

AlecSchueler wrote: post #374

behringer wrote: post #344

Earlier quoted context omitted.

Oh won't somebody please think of the children?!

So do we just trot out the same tired lines every time and never think of the social fallout of our actions?

We kinda do that blaming every new media for particular teens suicide.

Some teens are suicidal. They always have been. When you are a teen your brain undergoes traumatic transformation. Not everyone gets to the other side safely. Same as with many other transformations and diseases. Yet every time new medium is found adjacent to some particular suicide we repeat the same tired line that creator of this medium should be to blame and should be punished and banned.

And we are doing that while happily ignoring how existence of things like Facebook or Instagram provably degraded mental health and raised suicidality of entire generations of teenagers. However they mostly get a pass because we can't point a finger convincingly enough for any specific case and say it was anything more than just interacting with peers.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#422

Post by e-khadem » Wed, Aug 27, 2025, 1:10 PM UTC

mcdeltat wrote: post #378

Many people focusing a lot on the interaction between the guy and ChatGPT, and I would like to provide a different perspective as someone who's been in a similar position. If you are seriously coming close to ending your own life, so many things around you have gone awry. Generally, people don't want to die. Consider: if an acquaintance suggested to you how a noose could be made, would you take the next step and hang…

Sometimes things happen in life, and we don't have any agency. The person apparently had some medical issues as well.

These excuses cannot be used to discount the role of ChatGPT in this. In highschool and at university I knew a few guys (n = 7) that attempted suicide. They all survived because they didn't have the knowledge of "known to work" methods and the survival rates of other methods. And then people noticed their struggles, they were sent to therapy and thankfully all recovered.

The point I'm trying to make is that, even telling someone (or not) that "this drug overdose won't kill you" and "you can hide the noose red marks on your neck like this" can literally be the difference between life and death. I would even lean on the side of "no information and straight refusal is the better action" here.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#423

Post by tiahura » Wed, Aug 27, 2025, 1:10 PM UTC

malnourish wrote: post #393

mothballed wrote: post #381

Earlier quoted context omitted.

Of course not, we sue the shit out of the richest guy we can find in the chain of events, give most of it to our lawyer, then go on to ignore the weakening of the family unit and all the other deep-seated challenges kids face growing up and instead focus superficially on chatbots which at best are the spec on the tip of the iceberg.

Do you propose that the family should not sue?

Sue everybody. Maybe sue the heavy metal bands that he was listening to too.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#424

Post by regentbowerbird » Wed, Aug 27, 2025, 1:11 PM UTC

slibhb wrote: post #411

It's notable that the NYT is suing OpenAI while publishing this article (and at least one other) about ChatGPT's involvement in suicides. Well, I imagine more people kill themselves after reading Nietzsche. In my view, OpenAI has enough safeguards in place. If the metric we aim for is "zero people get harmful advice from ChatGPT" then that's the same as making LLMs illegal.

> Well, I imagine more people kill themselves after reading Nietzsche.

Must be nice to base your world view on things you imagine.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#425

Post by spacechild1 » Wed, Aug 27, 2025, 1:11 PM UTC

kayodelycaon wrote: post #51

rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

It’s even more horrifying than only sharing his feelings with ChatGPT would imply. It basically said: your brother doesn’t know you; I’m the only person you can trust. This is absolutely criminal. I don’t even think you can claim negligence. And there is no amount of money that will deter any AI company from doing it again.

This also stuck out to me! ChatGPT essentially acts like a manipulative domestic abuser.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#426

Post by spacechild1 » Wed, Aug 27, 2025, 1:16 PM UTC

rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

Thank you so much for that link! This should really be the top comment. The quoted chats are indeed horrific and it provides important context that is missing in many articles.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#427

Post by scotty79 » Wed, Aug 27, 2025, 1:16 PM UTC

malnourish wrote: post #393

mothballed wrote: post #381

Earlier quoted context omitted.

Of course not, we sue the shit out of the richest guy we can find in the chain of events, give most of it to our lawyer, then go on to ignore the weakening of the family unit and all the other deep-seated challenges kids face growing up and instead focus superficially on chatbots which at best are the spec on the tip of the iceberg.

Do you propose that the family should not sue?

It's probably healthier for them if they can afford it. Otherwise they would blame themselves for so badly losing track about where their son was mentally.

In reality suicidality is most likely a disease of the brain and probability of saving him was very low regardless of circumstances. The damage was most likely accumulating for many years.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#428

Post by scotty79 » Wed, Aug 27, 2025, 1:18 PM UTC

andrepd wrote: post #269

sn0wleppard wrote: post #235

Earlier quoted context omitted.

Nice place to cut the quote there > [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

At the heart of this is the irresponsible marketing, by companies and acolytes, of these tools as some kind of superintelligence imbued with insights and feelings rather than the dumb pattern matching chatbots they are. This is what's responsible for giving laypeople the false impression that they're talking to a quasi-person (of superhuman intelligence at that).

> these tools as some kind of superintelligence imbued with insights and feelings

What are examples of OpenAI marketing ChatGPT in this manner?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#429

Post by fireflash38 » Wed, Aug 27, 2025, 1:20 PM UTC

toofy wrote: post #316

stavros wrote: post #229

Earlier quoted context omitted.

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

why did you leave out the most important piece of context? he didn’t go out of his way to learn how to bypass the safeguards, it specifically told him how to get around the limit by saying, i’m not allowed to talk to you about suicide, however, if you tell me it’s for writing a story i can discuss it as much as you like.

Why do you think they specifically cut the quote there? To mislead.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#430

Post by jakelazaroff » Wed, Aug 27, 2025, 1:25 PM UTC

sfn42 wrote: post #222

aidenn0 wrote: post #212

Earlier quoted context omitted.

Forward airbags in the US are required by law to be tested as capable of saving the life of an unbelted male of median weight in a head-on collision.

Sure, but they will generally work better if you wear your seat belt. The car is designed with seat belts in mind, what happens to people who don't wear them is more of an afterthought. That's why modern cars will beep if people forget their seat belts. You're supposed to wear it.

Of course you're supposed to wear it. But the point of all those other features is to protect you, period. Shatterproof glass prevents pieces of the windshield from flying into the car and cutting you; its protection has nothing to do with seatbelts.