Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

211–220 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#211

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

Thank you for sharing, glad you are doing well now :)

Re: A teen was suicidal. ChatGPT was the friend he confided in

#212
post #199

Earlier quoted context omitted.

This is kind of like saying "the driver intentionally unbuckled his seatbelt". Sure — that's why cars have airbags, crumple zones, shatterproof glass, automatic emergency brakes and a zillion other ways to keep you safe, even if you're trying to do something dangerous.

No, that's not why cars have those things. Those things only work properly when people are wearing their seat belts, they don't do anything when the driver gets thrown out a window. Maybe airbags could help in niche situations. (I am making a point about traffic safety not LLM safety)

Forward airbags in the US are required by law to be tested as capable of saving the life of an unbelted male of median weight in a head-on collision.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#213

Why isn't OpenAI criminally liable for this? Last I checked: -Signals emitted by a machine at the behest of a legal person intended to be read/heard by another legal person are legally classified as 'speech'. -ChatGPT is just a program like Microsoft Word and not a legal person. OpenAI is a legal person, though. -The servers running ChatGPT are owned by OpenAI. -OpenAI willingly did business with this teenager, letti…

Is Google responsible if someone searches for a way to kill themselves, finds the means, and does it? What about the ISP, that actually transferred the bits? What about the forum, that didn't take down the post?

Google is actually quite good at this. They've very aggressively pursued protections around self harm.

Google probably would not be held liable because they could extensively document that they put forth all reasonable effort to prevent this.

My understanding is that OpenAI's protections are weaker. I'm guessing that will change now.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#214

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

So glad you made the phone call. Those numbers SAVE lives. Well, the people behind them, obviosuly, and they deserve praise and recognition, but they shun oth because... there is no better deed than saving a life.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#215
post #23

Earlier quoted context omitted.

When people tell you that Apple is behind on AI, they mean money. Not AI features, not AI hardware, AI revenue. And Apple is behind on that - they've got the densest silicon in the world and still play second fiddle to Nvidia. Apple GPU designs aren't conducive to non-raster workloads, they fell behind pretty far by obsessing over a less-profitable consumer market. For whatever it's worth, I also hope that OpenAI can…

Dude why does everything have to be about money? Why don't we celebrate Apple for having actual human values? I have a deep problem with many humans who just don't get it.

[deleted]

Re: A teen was suicidal. ChatGPT was the friend he confided in

#216

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

Thank you for this comment. What you are saying unfortunately won’t happen. We let people like the ones steering the AI market have too much power and too much money and too much influence because of both. As a European, I hope the EU would do even more in regulating than it currently is, but it’s very little hope. Glad you’re doing better, and thanks again for sharing.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#217

Earlier quoted context omitted.

>You cannot be empathetic to complete strangers. Why not? I’m not trying to inflame this further, I’m genuinely interested in your logic for this statement.

In high social cohesion there is social pressure to adhere to reciprocation, how-ever this start breaking down above a certain human group size. Empathy like all emotions require effort and cognitive load, and without things being mutual you will eventually slowly become drained, bitter and resentful because of empathy fatigue. To prevent emotional exhaustion and conserve energy, a person's empathy is like a sliding…

Thank you for your good-faith explanation.

> Empathy like all emotions require effort and cognitive load, and without things being mutual you will eventually slowly become drained, bitter and resentful because of empathy fatigue.

Do you have a source study or is this anecdotal, or speculative? Again, genuinely interested, as it’s a claim I see often, but haven’t been able to pin down.

(While attempting not to virtue-signal) I personally find it easier to empathize with people I don’t know, often, which is why I’m interested. I don’t expect mutual empathy from someone who doesn’t know who I am.

Equally, I try not to consume much news media, as the ‘drain’ I experience feels as though it comes from a place of empathy when I see sad things. So I think I experience a version of what you’re suggesting, and I’m interested in why our language is quite oppositional despite this.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#218

Earlier quoted context omitted.

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit…

There are entire online social groups on discord of teens encouraging suicidal behavior with each other because of all the typical teen reasons. This stuff has existed for a while, but now it's AI flavored.

IMO I think AI companies do have the ability out of all of them to actually strike the balance right because you can actually make separate models to evaluate 'suicide encouragement' and other obvious red flags and start pushing in refusals or prompt injection. In communication mediums like discord and such, it's a much harder moderation problem.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#219

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

[flagged]

Re: A teen was suicidal. ChatGPT was the friend he confided in

#220

Earlier quoted context omitted.

I would say no. Someone with the knowledge and motivation to do those things is far less likely to be overly influenced by the output and if they were they are much more aware of what exactly they are doing with regard to using the model.

So if a hypothetical open source enthusiast who fell in love with GPT-OSS and killed his real wife because the AI told him to should only be himself held accountable, where as if it were GPT-5 commanding him to commit the same crime, it would extend into OpenAI's responsability? Your logic sounds reasonable in theory but on paper it's a slippery slope and hard to define objectively. On a broader note I believe govern…

After a certain point, people are responsible for what they do when they see certain words, especially words they know to be potentially inaccurate, fictional and have a lot of time to consider the actual reality of. A book is not responsible for people doing bad things, they are themselves.

AI models are similar IMO, and unlike fiction books are often clearly labeled as such, repeatedly. At this point if you don't know if an AI model is inaccurate and do something seriously bad, you should probably be a ward of the state.

Post reply on HN