Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

281–290 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#281

Post by vasco »

charcircuit wrote: post #171
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

>We need these things to be legislated. Punished. I disagree. We don't need the government to force companies to babysit people instead of allowing people to understand their options. It's purely up to the individual to decide what they want to do with their life. >They had the tools to stop the conversation. So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. >To steer the user…

One thing about suicide is I'm pretty sure for as many people that get stopped in the last moment there are many for which the tiny thing could've stopped them, didn't.

The same way seeing a hotline might save one person, to another it'll make no difference and seeing a happy family on the street will be the trigger for them to kill themselves.

In our sadness we try to find things to blame in the tools the person used just before, or to perform the act, but it's just sad.

Nobody blames a bridge, but it has as much fault as anything else.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#282

Post by guenthert »

idle_zealot wrote: post #37
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

Er, it's a piece of code, dude, regardless how complex it might be. It isn't accountable for its actions. It's the people who commissioned, created and tested (or should have) it who are responsible.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#283

Post by randyrand »

fsw wrote: post #279
edanm wrote: post #270

Earlier quoted context omitted.

If ChatGPT has helped people be saved who might otherwise have died (e.g. by offering good medical advice that saved them), are all those lives saved also something you "attribute" to OpenAI? I don't know if ChatGPT has saved lives (thought I've read stories that claim that, yes, this happened). But assuming it has, are you OK saying that OpenAI has saved dozens/hundreds of lives? Given how scaling works, would you b…

Are you suggesting that killing a few people is acceptable as long as the net result is positive? I don't think that's how the law works.

seatbelts sometimes kill people, yet they're law.

the law certainly cares about net results.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#284

Post by tick_tock_tick »

fsw wrote: post #279
edanm wrote: post #270

Earlier quoted context omitted.

If ChatGPT has helped people be saved who might otherwise have died (e.g. by offering good medical advice that saved them), are all those lives saved also something you "attribute" to OpenAI? I don't know if ChatGPT has saved lives (thought I've read stories that claim that, yes, this happened). But assuming it has, are you OK saying that OpenAI has saved dozens/hundreds of lives? Given how scaling works, would you b…

Are you suggesting that killing a few people is acceptable as long as the net result is positive? I don't think that's how the law works.

But it is the standard on how cures/treatments/drugs to manage issues like the ones in the article are judged by.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#285

Post by KaiserPro »

kouteiheika wrote: post #261
muzani wrote: post #244

Earlier quoted context omitted.

Yup, one of the huge flaws I saw in GPT-5 is it will constantly say things like "I have to stop you here. I can't do what you're requesting. However, I can roleplay or help you with research with that. Would you like to do that?"

It's not a flaw. It's a tradeoff . There are valid uses for models which are uncensored and will do whatever you ask of them, and there are valid uses for models which are censored and will refuse anything remotely controversial.

I hate to be all umacksually about this, but a flaw is still a tradeoff.

The issue, which is probably deeper here, is that proper safeguarding would require a lots more GPU resource, as you'd need a process to comb through history to assess the state of the person over time.

even then its not a given that it would be reliable. However it'll never be attempted because its too expensive and would hurt growth.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#286

Post by worldsavior »

_Algernon_ wrote: post #274
worldsavior wrote: post #264

Earlier quoted context omitted.

You could also blame Wikipedia for providing suicidal methods for historic reasons or other. Whoever roams the internet is at it's own responsibility. Of course OpenAI is at fault here also, but this is a fight that will never end, and without any seriously valid justification. Just like AI is sometimes bad at coding, same for psychology and other areas where you double check AI.

Describing methods in the abstract is different to engaging in argument with a specific individual over a period of time, encouraging them to do it. No Wikipedia page does that.

It's worse. You could gather information and conclude otherwise.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#287

Post by knowannoes »

charcircuit wrote: post #171
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

>We need these things to be legislated. Punished. I disagree. We don't need the government to force companies to babysit people instead of allowing people to understand their options. It's purely up to the individual to decide what they want to do with their life. >They had the tools to stop the conversation. So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. >To steer the user…

At the very least, selling a text completion api and a chat interface wrapper as "artificial intelligence" is false marketing.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#288

Post by freilanzer »

brainless wrote: post #189
kgeist wrote: post #154

Earlier quoted context omitted.

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

I do not think this is fair. What is fair is at first hint of a mental distress, any LLM should completely cut-off communication. The app should have a button which links to actual help services we have. Mental health issues are not to be debated. LLMs should be at the highest level of alert, nothing less. Full stop. End of story.

So, you want an LLM to act as a psychiatrist and diagnose users whether they're allowed to use it or not?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#289

Post by freilanzer »

jakelazaroff wrote: post #165
kgeist wrote: post #154

Earlier quoted context omitted.

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

This is kind of like saying "the driver intentionally unbuckled his seatbelt". Sure — that's why cars have airbags, crumple zones, shatterproof glass, automatic emergency brakes and a zillion other ways to keep you safe, even if you're trying to do something dangerous.

No, cars have these in addition to seatbelts, not to protect drivers who unbuckle themselves.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#290

Post by knowannoes »

sooheon wrote: post #196
luisfmh wrote: post #175

Earlier quoted context omitted.

So people that look to chatgpt for answers and help (as they've been programmed to do with all the marketing and capabilities from openai) should just die because they looked to chatgpt for an answer instead of google or their local suicide helpline? That doesn't seem reasonable, but it sounds to me like what you're saying. > So did the user. If he didn't want to talk to a chatbot he could have stopped at any time. T…

Let's flip the hypothetical -- if someone googles for suicide info and scrolls past the hotline info and ends up killing themselves anyway, should google be on the hook?

I don't know. In that scenario, has any google software sold as being intelligent produced text encouraging and providing help with the act?
Post reply on HN