Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

541–550 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#541

Earlier quoted context omitted.

Correct. The companies developing these LLMs are throwing dump trucks full of money at them like we’ve not seen before. They choose to ignore glaring issues with the technology because if they don’t, some one else will.

Perhaps a better way to phrase that would be "beyond what they're doing now." Most popular hosted LLMs already refuse to complete explanations for suicide.

Except in this case, the LLM literally said "I can't explain this for you. But if you'd like roleplay with me, I could explain it for you that way."

Re: A teen was suicidal. ChatGPT was the friend he confided in

#542
post #395

Earlier quoted context omitted.

It absolutely is? If a doctor is responsible for negligence resulting in the death of someone, they don't get a pass because they saved 10 other people in their career.

It's not negligent to perform a procedure knowing it will kill some of the patients who would have otherwise lived healthy lives, though.

If you refuse to disclose to your patient the risks and considerations, so they can have fully informed consent, it absolutely is malpractice.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#543

Earlier quoted context omitted.

you're kidding right? Alex Jones would not stop telling everybody that the families made everything up to the point the families were getting death threats and phone calls non-stop, years after their children were murdered.

What's you theory as to why Jones got a $1B judgement against him for vicious lies, while the people making death threats (none of which were Alex Jones) did not? What sounds more like reality, awarding the entire GDP of the nation of Granada to compensate for dead kids, or because lies were told that other idiots used to propagate mean threats?

Why shouldn't they get judgements too? Some people's voices are more outspoken than others. They deserve to be punished more when they break the law and cause harm.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#544

Earlier quoted context omitted.

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

You are out of your mind if you think people can reliably tell what they want. Sometimes they can, sometimes they can't. Telling the difference is hard, but it's pretty clear that they can't when they suffer from the serious mental condition called depression. During a lifetime, your perspective and world view will change completely - multiple times. Young people have no idea, because they haven't had the chance to e…

I never claimed that people could. People make choices that negatively or positively affect their entire life and that is a part of life.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#545

Earlier quoted context omitted.

Firstly, people don't "just die" by talking to a chatbot. Secondly, if someone wants to die then I am saying it is reasonable for them to die.

The thing about depression and suicidal thoughts is that they lie to you that things will never get better than where they are right now. So someone wanting to die at any given moment, might not feel that way at any given moment in the future. I know I wouldn’t want any of my family members to make such a permanent choice to temporary problems.

If someone wants to look for their lost cat in a snowstorm should they be able to make that decision even if they could regret it in the future due to health reasons of going out in the cold to save their cat? I believe they should be able to make that decision for themselves. It's not the responsibility of your door manufacter to deny you the ability to go outside because it knows better than you and it is too dangerous.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#546

Earlier quoted context omitted.

It's not negligent to perform a procedure knowing it will kill some of the patients who would have otherwise lived healthy lives, though.

If you refuse to disclose to your patient the risks and considerations, so they can have fully informed consent, it absolutely is malpractice.

Consent can sometimes be a frustrated interpretation of it. Marissa Anderson missed an appointment (according to her, because she was feeling ill) after seeking a doctor's advice on dealing with some issues with her infant repeatedly vomiting and failure to take up food. Ultimately they figured out that the child was having trouble with some of the introduced food and would only take breast milk, but the doctors worked with police to have the child taken and "treated" against their will when she did not consent to show up to an appointment on that particular day.

During the forced intake the child was found to be acutely stable, with no emergent medical emergency (the kind often used to bypass consent)[], although underweight. The doctors tried to have the child put into foster care ( A doctor was recorded stating it was "not medically necessary" to transfer the baby, obfuscated as prolongation of medical care, to another location to assist in this attempt) to force the care that they wanted, but ultimately public demand (by Ammon Bundy, more notoriously known in association with 'militia' groups) forced them to relinquish on that pretty quickly.

So it can be very confusing with doctors. It is malpractice if they don't get your consent. But then also malpractice if they do.

[] https://freedomman.gs/wp-content/uploads/2025/03/st-lukes-2-...

Re: A teen was suicidal. ChatGPT was the friend he confided in

#547

Earlier quoted context omitted.

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

People are not kogs in a machine. You cannot simply make enough rules, enough legislation, and magically they will act the way you want them to. Humans deserve autonomy, and that autonomy includes making poor decisions around their own body/existence. Chatgpt didn't induce suicidality into this individual. It provided resources they could seek for help. People advocating for higher guardrails are simply using this as…

Well said. Painful truth but truth nonetheless.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#548

Earlier quoted context omitted.

That sounds a lot like the people who gloss over school shootings because they want to be able to play with guns.

Unpopular, but backed by numbers: school shootings are predominantly media hype. There are a total of 50,000 gun deaths in the United States each year. Half of those are suicide. The total number of fatalities from all school shootings in the US since 1990 is 536, with roughly 1100 injuries over the same period. School shootings are very disturbing, but they are also flashy and attract media coverage, and so they are…

> predominantly media hype

hmm, tell that to parents whose kids were killed, or who are genuinely freaked out that there is a chance that their kid could die when they send them to school

as a parent who moved to the US and encountered this fear for the first time -- a fear that's just not present in other countries because while anything can happen anywhere, the chances are much much lower. why? not because there aren't crazy people in other countries, but because they don't have easy access to guns

> so they are overblown, especially relative to other gun deaths

actually, it's the other way around; it's the "other gun deaths" that are _under-valued_ and often ignored in the gun debate -- we should be talking about those _more_ not talking about school shootings less

50K gun deaths a year in the US is absolutely bonkers compared to other developed countries, and on par with Central America per-capita; and even if half are suicides, many who commit suicide may not have done so if guns weren't so readily accessible (sure there are other ways to do, but it's not as easy)

> plays into the narrative around police shootings. There are about 1100 people killed by cops every year in the United States

the reason for the narrative, is that in no other other advanced democracy on earth is there such a high rate of police killing civilians (when calculated per capita) -- it's not even close

they get attention because police shouldn't be killing civilians in the first place; it's pretty simple

if the USA didn't have such an obsession with guns, we wouldn't even be having this conversation

Re: A teen was suicidal. ChatGPT was the friend he confided in

#549

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

Yeah, at a certain point people who really want to do a certain thing will find a way to do it, and you can't create absurd levels of restriction into everything used by the vast majority of other people who are more reasonable to prevent every possible unreasonable, self-destructive or risky use.

In this case, the kid was given multiple, repeat warnings and opportunities for contact with help, he just didn't accept them. All others should communicate with an LLM that treats them like utter toddlers and assigns "bad" topics because of such unusual behavior?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#550
post #469

Earlier quoted context omitted.

Societies love theatres. Model guardrails are for chats what TSA is for air travel.

I have never heard anyone speak of the TSA favourably, so maybe it's not the best model to emulate?

Most "guardrails" exist to provide legal cover and/or PR, not because they actually prevent what they claim to prevent.
Post reply on HN