Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

401–410 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#401

Post by podgietaru »

itvision wrote: post #379
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> We are living in such a horrific moment. We need these things to be legislated. Punished. We need to stop treating them as magic. They had the tools to prevent this. They had the tools to stop the conversation. To steer the user into helpful avenues. No, no, no and no. ChatGPT wasn't the source of his desire to end his life, nor was it the means to do it. It was a "person" to talk to, since he had no such real peop…

I clearly didn't claim that. My own depression was multifaceted, stemming from genetics, upbringing, social isolation. And the help I got was also multifaceted and involved identifying all those causes.

Does it not give you pause when you write things like "it's just ChatGPT. Your own attempt at ending your life hasn't seemingly taught you anything."

Suicidality is multi-faceted. It's why there's recorded records of suicide attempts increasing when a highly-publicised suicide attempt happens, or when media portrays it such as in 13 Reasons why.

And when you're straddling that line - as you might imagine - your sense of rationality is sort of all over the place.

As with probably a lot of other influences in his life the responses of ChatGPT probably tugged him in a direction. And that alone is horrific.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#402

Post by UltraSane »

ceejayoz wrote: post #170
UltraSane wrote: post #145

Earlier quoted context omitted.

"It sounds like you're mostly just talking to yourself" No, Claude does know a LOT more than I do about most things and does push back on a lot of things. Sometimes I am able to improve my reasoning and other times I realize I was wrong. Trust me, I am aware of the linear algebra behind the curtain! But even when you mostly understand how they work the best LLMs today are very impressive. And latent spaces fundamenta…

> No, Claude does know a LOT more than I do about most things… Plenty of people can confidently act like they know a lot without really having that knowledge.

So you are denying that LLMs actually contain real knowledge?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#403

Post by cameronh90 »

podgietaru wrote: post #339
cameronh90 wrote: post #336

Earlier quoted context omitted.

I'm a happy sertraline user, but your experience sounds like the exception. Most people are prescribed antidepressants by their GP/PCP after a short consultation. In my case, I went to the doctor, said I was having problems with panic attacks, they asked a few things to make sure it was unlikely to be physical and then said to try sertraline. I said OK. In and out in about 5 minutes, and I've been on it for 3 years n…

This is probably fair - My experience comes both from the UK (where it was admittedly worse, but not that much) and the Netherlands - where it was fantastic. Even in the worst experiences, I had a followup appointment in 2, 4 and 6 weeks to check the medication.

My experience is in the UK, but it doesn't surprise me that you got more attention in the Netherlands. From the experiences of my family, if you want anything more than a paracetamol, you practically need sign off from the Minister of Health!

Joking aside, they do seem to escalate more to specialists whereas we do more at the GP level.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#404

Post by podgietaru »

behringer wrote: post #400
footy wrote: post #382

Earlier quoted context omitted.

Have you read the chat logs? Just asking because ChatGPT specifically encouraged this kid not to seek help.

ChatGPT is not a human, it can't know if it's doing the right thing or not. The parents should have been monitoring his usage and teaching him about LLMs.

OpenAI has the ability to detect whether a conversation is about a certain topic. It has the ability to end a conversation, or, if you think that is too much, it has the ability to prominently display information.

My preference would be that in the situation that happened in the story above that it would display a prominent banner ad above the chat with text akin to.

"Help and support is available right now if you need it. Phone a helpline: NHS 111. Samartians.. Etc.

ChatGPT is a chatbot, and is not able to provide support for these issues. You should not follow any advice that ChatGPT is offering.

We suggest that you:

Talk to someone you trust: Like family or friends.

Who else you can contact:

* Call a GP, * Call NHS 111 etc "

This ad should be displayed at the top of that chat, and be undismissable.

The text it offered is so far away from that it's unreal. And the problem with these chatbots is absolutely a marketing one. Because they're authoritative, and presented as emotional and understanding. They are not human, as you said. But the creators don't mind if you mistake them as such.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#405

Post by robhlt »

kouteiheika wrote: post #261
muzani wrote: post #244

Earlier quoted context omitted.

Yup, one of the huge flaws I saw in GPT-5 is it will constantly say things like "I have to stop you here. I can't do what you're requesting. However, I can roleplay or help you with research with that. Would you like to do that?"

It's not a flaw. It's a tradeoff . There are valid uses for models which are uncensored and will do whatever you ask of them, and there are valid uses for models which are censored and will refuse anything remotely controversial.

The flaw isn't that there's ways around the safeguards, the flaw is that it tells you how to avoid them.

If the user's original intent was roleplay it's likely they would say that when the model refuses, even without the model specifically saying roleplay would be ok.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#406

Post by franktankbank »

kouteiheika wrote: post #261
muzani wrote: post #244

Earlier quoted context omitted.

Yup, one of the huge flaws I saw in GPT-5 is it will constantly say things like "I have to stop you here. I can't do what you're requesting. However, I can roleplay or help you with research with that. Would you like to do that?"

It's not a flaw. It's a tradeoff . There are valid uses for models which are uncensored and will do whatever you ask of them, and there are valid uses for models which are censored and will refuse anything remotely controversial.

This is one model though. "I'm sorry I'm censored but if you like I can cosplay quite effectively as an uncensored one." So you're not censored really?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#407

Post by podgietaru »

unnamed76ri wrote: post #348
charcircuit wrote: post #303

Earlier quoted context omitted.

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

The thing about depression and suicidal thoughts is that they lie to you that things will never get better than where they are right now. So someone wanting to die at any given moment, might not feel that way at any given moment in the future. I know I wouldn’t want any of my family members to make such a permanent choice to temporary problems.

1000% As I said in my comment. I never thought I'd be better. I am. I am happy and I live a worthwhile life.

In the throws of intense depression it's hard to even wake up. The idea that I was acting in my right mind, and was able to make a decision like that is insane to me.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#408

Post by causal »

incone123 wrote: post #157
causal wrote: post #104

Earlier quoted context omitted.

It is disturbing, but I think a human therapist would also have told him not to do that, and instead resorted to some other intervention. It is maybe an example of why having a partial therapist is worse than none: it had the training data to know a real therapist wouldn't encourage displaying nooses at home, but did not have the holistic humanity and embodiment needed to intervene appropriately. Edit: I should add t…

Even here you are anthropomorphising. It doesn't 'know' anything. A human therapist would escalate this to a doctor or even EMS.

"know" is used colloquially this way even for non-AI systems.

"it encodes the training data in weights to predict a token mimicking a human ..." - better?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#409

Post by mathiaspoint »

broker354690 wrote: post #109

Why isn't OpenAI criminally liable for this? Last I checked: -Signals emitted by a machine at the behest of a legal person intended to be read/heard by another legal person are legally classified as 'speech'. -ChatGPT is just a program like Microsoft Word and not a legal person. OpenAI is a legal person, though. -The servers running ChatGPT are owned by OpenAI. -OpenAI willingly did business with this teenager, letti…

What's your argument here? Hosted LLM service shouldn't exist because they might read people's bad ideas back to them?

ChatGPT has enough guardrails now that it often refuses productive prompts. It's actually very very hard to get it to do what this person did, arguably impossible to do unintentionally.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#410

Post by scotty79 »

podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> We need these things to be legislated. Punished.

That's harsh. Way more people in history killed themselves because they read a moody book than because interaction with AI pushed them in this direction. Werther effect.

How many suicides is music "responsible" for? Movies? Theatre? Clubbing? Friends?

You can't legislate people out of being danger to themselves by punishing all the possible triggers.

Post reply on HN