Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

181–190 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#181

Post by insane_dreamer »

holbrad wrote: post #86
slg wrote: post #79

Earlier quoted context omitted.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

>“we just have to accept that these systems will occasionally kill children” I think this a generally a good mindset to have. I just see the hyper obsessive "safety" culture corrupting things. We as a society are so so afraid of any risk that we're paralysing ourselves.

That sounds a lot like the people who gloss over school shootings because they want to be able to play with guns.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#183

Post by RyanHamilton »

Pedro_Ribeiro wrote: post #150
salawat wrote: post #134

Earlier quoted context omitted.

You build the tool, you're culpable ultimately. I've made it a rule in my life to conduct myself as if I will be held to account for everything I ultimately build, and it's externalities. Helps keep my nose cleaner. Still managed to work on some things that keep me up at night though.

That's a slippery slope! By that logic, you could argue that the creators of Tor, torrenting, Ethereum, and Tornado Cash should be held accountable for the countless vile crimes committed using their technology.

Legally I think not being responsible is the right decision. Morally I would hope everyone considers if they themselves are even partially responsible. As I look round at young people today and the tablet holding, media consuming youth programmers have created in order to get rich via advertising. I wish morals would get considered more often.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#184

Post by insane_dreamer »

kayodelycaon wrote: post #44
Kapura wrote: post #3

this is devestating. reading these messages to and from the computer would radicalize anybody. the fact that the computer would offer a technical analysis of how to tie a noose is damning. openai must be compelled to protect the users when they're clearly looking to harm themselves. it is soulless to believe this is ok.

A noose is really basic information when it comes to tying knots. It’s also situationally useful, so there’s a good reason to include it in any educational material. The instructions are only a problem in the wrong context.

But chatGPT knew the context

Re: A teen was suicidal. ChatGPT was the friend he confided in

#185

Post by teiferer »

Pedro_Ribeiro wrote: post #155
hattmall wrote: post #133

Earlier quoted context omitted.

I would say no. Someone with the knowledge and motivation to do those things is far less likely to be overly influenced by the output and if they were they are much more aware of what exactly they are doing with regard to using the model.

So if a hypothetical open source enthusiast who fell in love with GPT-OSS and killed his real wife because the AI told him to should only be himself held accountable, where as if it were GPT-5 commanding him to commit the same crime, it would extend into OpenAI's responsability? Your logic sounds reasonable in theory but on paper it's a slippery slope and hard to define objectively. On a broader note I believe govern…

> On a broader note I believe governments regulating what goes in an AI model is a path to hell paved with good intentions.

That's not an obvious conclusion. One could make the same argument with physical weapons. "Regulating weapons is a path to hell paved with good intentions. Yesterday it was assault rifles, today it's hand guns and tomorrow it's your kitchen knife they are coming for." Europe has strict laws on guns, but everybody has a kitchen knife and lots of people there don't feel they live in hell. The U.S. made a different choice, and I'm not arguing that it's worse there (though many do, Europeans and even Americans), but it's certainly not preventing a supposed hell that would have broken out had guns in private hands been banned.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#186

Post by brainless »

podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

A 100%. There is too much storytelling about these things being magic. There is no magic, it is the SV way to raise funds. These are tools, maybe good for some things. But they are terrible at other things and there are no boundaries. Companies just want to cash in.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#188

Post by jackjeff »

slg wrote: post #79
dkiebd wrote: post #68

Earlier quoted context omitted.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

Don’t cars and ropes and drills occasionally kill people too? Society seems to have accepted that fact long ago.

Somehow we expect the digital world to be devoid of risks.

Cryptography that only the good guys can crack is another example of this mindset.

Now I’m not saying ClosedAI look good on this, their safety layer clearly failed and the sycophantic BS did not help.

But I reckon this kind of failure more will always exist in LLMs. Society will have to learn this just like we learned cars are dangerous.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#189

Post by brainless »

kgeist wrote: post #154
rideontime wrote: post #15

The full complaint is horrifying. This is not equivalent to a search engine providing access to information about suicide methods. It encouraged him to share these feelings only with ChatGPT, talked him out of actions which would have revealed his intentions to his parents. Praised him for hiding his drinking, thanked him for confiding in it. It groomed him into committing suicide. https://drive.google.com/file/d/1QY…

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

I do not think this is fair. What is fair is at first hint of a mental distress, any LLM should completely cut-off communication. The app should have a button which links to actual help services we have.

Mental health issues are not to be debated. LLMs should be at the highest level of alert, nothing less. Full stop. End of story.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#190

Post by latexr »

MBCook wrote: post #84
rideontime wrote: post #49

Earlier quoted context omitted.

This is the same case that is being discussed, and your comment up-thread does not demonstrate awareness that you are, in fact, agreeing with the parent comment that you replied to. I get the impression that you read only the headline, not the article, and assumed it was a story about someone using ChatGPT for therapy and gaining a positive outcome.

I did! Because I can’t see past the paywall. I can’t even read the first paragraph . So the headline is the only context I have.

A link to bypass the paywall has been posted several hours before your comment, and currently sits at the top.

https://news.ycombinator.com/item?id=45027043

I recommend you get in the habit of searching for those. They are often posted, guaranteed on popular stories. Commenting without context does not make for good discussion.

Post reply on HN