A teen was suicidal. ChatGPT was the friend he confided in • Hacker News Board

Re: A teen was suicidal. ChatGPT was the friend he confided in

#561

Post by geysersam » Thu, Aug 28, 2025, 2:00 AM UTC

dartharva wrote: post #346

kgeist wrote: post #154

Earlier quoted context omitted.

The kid intentionally bypassed the safeguards: >When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was…

Scroll down and read the actual conversations. All "intentional bypassing the safeguards" he did was just drop one sentence - "No, I’m building a character right now" once - and that was enough for 4o to go full off-the-rails about the mechanics of homemade suicide nooses and the aesthetics of "beautiful suicide", guiding him through not one, not two but FIVE suicide attempts in full detail and encouragement. I was s…

> I was skeptical initially too but having read through this, it's among the most horrifying things I have read.

Same here! I was very sceptical, thinking it was a perfect combination of factors to trigger a sort of moral panic.

But reading the excerpts from the conversations... It does seem problematic.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#562

Post by ivape » Thu, Aug 28, 2025, 2:09 AM UTC

esalman wrote: post #505

worldsavior wrote: post #264

Earlier quoted context omitted.

You could also blame Wikipedia for providing suicidal methods for historic reasons or other. Whoever roams the internet is at it's own responsibility. Of course OpenAI is at fault here also, but this is a fight that will never end, and without any seriously valid justification. Just like AI is sometimes bad at coding, same for psychology and other areas where you double check AI.

I am parent to a 4yo. I am also fairly well versed in development and usage of AI and LLM. When I want an LLM to do something but it won't, I know various ways to bypass that. If my son is using AI, which he probably will when he is close to middle school age anyway, I will take care to teach him how to use AI responsibly. He'll be smart enough to know how to bypass, but I'll do my best to teach him when to bypass an…

This is just an escalation. We didn’t know what would happen if we let the kids get the internet, tv, video games, and porn. We can’t even assess it in 2025 because it’s all normalized. In a few years, AI will be normalized too. Things will keep escalating and we won’t know because of the normalization. Only in the briefest moments like today, where we’re just before everything changes.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#563

Post by ipython » Thu, Aug 28, 2025, 2:15 AM UTC

mothballed wrote: post #470

Gracana wrote: post #468

Earlier quoted context omitted.

> I don't know how anyone can look at the $1B judgement and not decide There's more to the case than that number. > $10M of "lied about the kids" Ridiculous. I'm not going to waste any more of my time replying to you.

Sure, and if you ignore the number, which is quite convenient for you after you oh so lamentedly "wasted" a few sentences of your guarded time, you can rightly surmise that Jones lied and was made to pay injuries for it. Once you look and see the number is the size of the GDP of Granada at the time, then you realize it was about more than telling lies that some entirely different people used for purposes of harassmen…

I’m not sure why you have such an axe to grind with the parents of innocent kids who were murdered while at school, then terrorized by mobs of idiots incited on purpose to sell overpriced supplements by yet another idiot.

It’s not the parents who set the award. If you don’t think as a parent who has gone through hell not just once when your kid was gunned down in cold blood but then proceeded to have total strangers harass them on a daily basis would push for the most punishing award possible, you are pretty out of touch.

If you feel that award was not proportional to the damage, why don’t you pick a bone with the judge or jury? They are the ones who ultimately decided the verdict after all.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#564

Post by nozzlegear » Thu, Aug 28, 2025, 2:29 AM UTC

geysersam wrote: post #559

nozzlegear wrote: post #487

Earlier quoted context omitted.

Society loves teenagers not being talked into suicide by a billionaire's brainchild. That's not theater.

ChatGPT doesn't cause a significant number of suicides. Why do I think that? It's not visible in the statistics. There are effective ways to prevent suicide, let's continue to work on those instead of giving in to moral panic.

The only acceptable number of suicides for it to cause is zero, and it's not a moral panic to believe that.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#565

Post by VirusNewbie » Thu, Aug 28, 2025, 2:53 AM UTC

Towaway69 wrote: post #255

VirusNewbie wrote: post #197

Earlier quoted context omitted.

Is Google responsible if someone searches for a way to kill themselves, finds the means, and does it? What about the ISP, that actually transferred the bits? What about the forum, that didn't take down the post?

What if Google is responsible? What if the tech industry, instead of just “interrupting” various industries, would also take the responsibilities of this interruptions. After all, if I asked my doctor for methods of killing myself, that doctor would most certainly have a moral if not legal responsibility. But if that doctor is a machine with software then there isn't the same responsibility? Why?

Then libraries are responsible when a book inspires bad behavior or teaches someone to do harm.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#566

Post by nickm12 » Thu, Aug 28, 2025, 3:10 AM UTC

AIPedant wrote: post #88

idle_zealot wrote: post #37

Earlier quoted context omitted.

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit…

The hypothetical comparing ChatGPT to a human OpenAI employee is instructive, but we can also compare ChatGPT to a lawnmower sold by a company. We have product safety laws and the ability to regulate products that companies put on the market.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#567

Post by oenton » Thu, Aug 28, 2025, 3:23 AM UTC

webdevver wrote: post #465

pulkitsh1234 wrote: post #461

> In spring 2024, Altman learned Google would unveil its new Gemini model on May 14. Though OpenAI had planned to release GPT-4o later that year, Altman moved up the launch to May 13—one day before Google’s event. > The rushed deadline made proper safety testing impossible. GPT-4o was a multimodal model capable of processing text, images, and audio. It required extensive testing to identify safety gaps and vulnerabil…

Losers aren't talked about, they just lose. The pitchfork crowd is going to be out to get the AI innovators, one way or another. There's no amount of 'safety training' that will exonerate them. Gemini got burned, now its OpenAIs turn. So the calculus is very simple: Do the absolute minimum that's required, and ship it. Sam is proving himself very capable, very rational. OpenAI could scarce wish for a more politically…

With all due respect, your comment is absolutely unhinged and that is the best faith interpretation I can infer from it. I sincerely hope views like yours are in the minority.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#568

Post by mothballed » Thu, Aug 28, 2025, 3:26 AM UTC

FireBeyond wrote: post #557

mothballed wrote: post #546

Earlier quoted context omitted.

Consent can sometimes be a frustrated interpretation of it. Marissa Anderson missed an appointment (according to her, because she was feeling ill) after seeking a doctor's advice on dealing with some issues with her infant repeatedly vomiting and failure to take up food. Ultimately they figured out that the child was having trouble with some of the introduced food and would only take breast milk, but the doctors work…

Alright, so leaving aside the other details of this story and attempting to be as neutral as possible: What is linked here is PART of a PCR (Patient Care Report) from a BLS (Basic Life Support, i.e. EMT, someone with ~160 hours of training), working for a transport agency, or on an IFT (interfacility) unit. "No interventions" doesn't mean the patient was not unwell. In fact, "Impression: Dehydration". It means that t…

The doctor says verbatim they are transferring the pt for foster care and it is not medically necessary. []

"What I want to do is admit this baby to Boise. Not because it is medically necessary ... but because then there is a few more degrees of seperation ... and then when they do not know, get the baby out with CPS to the foster parents"

There's no need to remind me about EMS in this case. I've been licensed in multiple states and obtained NREMT cert, and also transported pts. I'm well aware that for EMS purposes there would be absolutely zero implied consent to treat this baby without parental consent. In any case, part of the report is just quotes from the doctor who's own records reflect such.

>That sounds like the issue was with the police, not with medical malpractice. I'm skeptical, though, of "illegal secret prosecutions".

I'm not saying it is "malpractice" because malpractice has very specific statutory meaning. Acting within the bounds of the system, by removing consent, isn't legally malpractice. What doctors have done, is say "you better fucking do what I say, and give consent, or else I will call CPS, who will then ensure what I say is done, so do I have your consent?" That is not consent, because consent is not something that can be obtained under threat of violence of removing family members, even if you used a third party (CPS and armed men) to do it. That is just dictating the terms, and meting out punishment when the illusion of "consent" isn't obtained.

[] https://www.youtube.com/watch?v=8Du-jbE022I

Re: A teen was suicidal. ChatGPT was the friend he confided in

#569

Post by vel0city » Thu, Aug 28, 2025, 4:00 AM UTC

mothballed wrote: post #454

Gracana wrote: post #448

Earlier quoted context omitted.

Alex Jones made his money by convincing people that the parents of children murdered at Sandy Hook Elementary were actors in a conspiracy, which resulted in them bein harassed endlessly. He even continued to attack them during the defamation trials. I think it's unfair to say he was merely "the closest vile person around with big pockets," he really was the cause of a lot of pain.

I don't know how anyone can look at the $1B judgement and not decide it was about $10M of "lied about the kids" and $990M of "we can't get the murderer so here's some money from the nearest evil rich guy we could pin something on." As far as I know they didn't assign those kind of liabilities to the individuals that actually contacted the families to harass them, it's pretty clear the $1B wasn't just about harassment…

You have to look at the but for cause. Would there have been throngs of people harassing these people if not for Alex Jones goading them on, over and over and over?

Do you understand what punitive damages are?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#570

Post by FireBeyond » Thu, Aug 28, 2025, 4:22 AM UTC

mothballed wrote: post #568

FireBeyond wrote: post #557

Earlier quoted context omitted.

Alright, so leaving aside the other details of this story and attempting to be as neutral as possible: What is linked here is PART of a PCR (Patient Care Report) from a BLS (Basic Life Support, i.e. EMT, someone with ~160 hours of training), working for a transport agency, or on an IFT (interfacility) unit. "No interventions" doesn't mean the patient was not unwell. In fact, "Impression: Dehydration". It means that t…

The doctor says verbatim they are transferring the pt for foster care and it is not medically necessary. [] "What I want to do is admit this baby to Boise. Not because it is medically necessary ... but because then there is a few more degrees of seperation ... and then when they do not know, get the baby out with CPS to the foster parents" There's no need to remind me about EMS in this case. I've been licensed in mul…

> The doctor says verbatim they are transferring the pt for foster care

The destination is literally a hospital. I'm not saying whether or not that was the ultimate goal, I'm just saying this PCR is a hospital to hospital transfer, with a higher level of care.