Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

231–240 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#231

Post by arcticbull »

aidenn0 wrote: post #205
AIPedant wrote: post #88

Earlier quoted context omitted.

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit…

IANAL, but: One first amendment test for many decades has been "Imminent lawless action." Suicide (or attempted suicide) is a crime in some, but not all states, so it would seem that in any state in which that is a crime, directly inciting someone to do it would not be protected speech. For the states in which suicide is legal it seems like a much tougher case; making encouraging someone to take a non-criminal action…

> Which is almost certainly protected speech (just as any odious white-nationalist propaganda is protected).

Fun fact, much of the existing framework on the boundaries of free speech come from Brandenburg v. Ohio. You probably won't be surprised to learn that Brandenburg was the leader of a local Klan chapter.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#232

Post by blackqueeriroh »

AIPedant wrote: post #88
idle_zealot wrote: post #37

Earlier quoted context omitted.

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

Yes, if this were an adult human OpenAI employee DMing this stuff to a kid through an official OpenAI platform, then a) the human would (deservedly[1]) be arrested for manslaughter, possibly murder b) OpenAI would be deeply (and deservedly) vulnerable to civil liability c) state and federal regulators would be on the warpath against OpenAI Obviously we can't arrest ChatGPT. But nothing about ChatGPT being the culprit…

Section 230 changes 2) and 3). OpenAI will argue that it’s user-generated content, and it’s likely that they would win.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#233

Post by blackqueeriroh »

notachatbot123 wrote: post #226
idle_zealot wrote: post #37

Earlier quoted context omitted.

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

I so agree very much. There is no reason for LLMs to be designed as human-like chat companions, creating a false sense of untechnology.

There are absolutely reasons for LLMs to be designed as human-like chat companions, starting with the fact that they’re trained on human speech and behavior, and what they do is statistically predict the most likely next token, which means they will statistically sound and act much like a human.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#234

Post by wolvesechoes »

VirusNewbie wrote: post #197
broker354690 wrote: post #109

Why isn't OpenAI criminally liable for this? Last I checked: -Signals emitted by a machine at the behest of a legal person intended to be read/heard by another legal person are legally classified as 'speech'. -ChatGPT is just a program like Microsoft Word and not a legal person. OpenAI is a legal person, though. -The servers running ChatGPT are owned by OpenAI. -OpenAI willingly did business with this teenager, letti…

Is Google responsible if someone searches for a way to kill themselves, finds the means, and does it? What about the ISP, that actually transferred the bits? What about the forum, that didn't take down the post?

Driver that shipped alcohol to the store is not responsible for the fact that clerk sold it to some kid. Clerk still is.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#235

Post by sn0wleppard »

stavros wrote: post #229
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

Nice place to cut the quote there

> [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

Re: A teen was suicidal. ChatGPT was the friend he confided in

#236

Post by latexr »

novok wrote: post #227
latexr wrote: post #203

Earlier quoted context omitted.

> I think the mom's reaction of "ChatGPT killed my son" is ridiculous: no, your son killed himself. ChatGPT facilitated it (…) That whole paragraph is quite something. I wonder what you’d do if you were given the opportunity to repeat those words in front of the parents. I suspect (and hope) some empathy might kick in and you’d realise the pedantry and shilling for the billion dollar company selling a statistical wor…

You do have empathy for the person who had a tragedy, but it doesn't mean you go into full safetyism / scapegoating that causes significantly less safety and far more harm because of the emotional weight of something in the moment. It's like making therapists liable for people committing suicide or for people with eating disorders committing suicide indirectly. What ends up happening when you do is therapists avoidin…

That is one big slippery slope fallacy. You are inventing motives, outcomes, and future unproven capabilities out of thin air. It’s a made up narrative which does not reflect the state of the world and requires one to buy into a narrow, specific world view.

https://en.wikipedia.org/wiki/Slippery_slope

Re: A teen was suicidal. ChatGPT was the friend he confided in

#237

Post by Dilettante_ »

insane_dreamer wrote: post #181
holbrad wrote: post #86

Earlier quoted context omitted.

>“we just have to accept that these systems will occasionally kill children” I think this a generally a good mindset to have. I just see the hyper obsessive "safety" culture corrupting things. We as a society are so so afraid of any risk that we're paralysing ourselves.

That sounds a lot like the people who gloss over school shootings because they want to be able to play with guns.

Wow. Incredible framing, leaving absolutely no room for a rebuttal to have any legitimate, reasonable position.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#238

Post by blackqueeriroh »

slipperydippery wrote: post #98
idle_zealot wrote: post #76

Earlier quoted context omitted.

No, that's the level of responsibility they ought to have if they were releasing these models as products. As-is they've used a service model, and should be held to the same standards as if there were a human employee on the other end of the chat interface. Cut through the technical obfuscation. They are 100% responsible for the output of their service endpoints. This isn't a case of making a tool that can be used fo…

I like this framing even better. This is similar to my take on things like Facebook apparently not being able to operate without psychologically destroying moderators. If that’s true… seems like they just shouldn’t operate , then. If you’re putting up a service that you know will attempt to present itself as being capable of things it isn’t… seems like you should get in a shitload of trouble for that? Like maybe don’…

But understand that things like Facebook not operating doesn’t actually make the world any safer. In fact, it makes it less safe, because the same behavior is happening on the open internet and nobody is moderating it.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#239

Post by [deleted] »

slg wrote: post #79
dkiebd wrote: post #68

Earlier quoted context omitted.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

[deleted]

Re: A teen was suicidal. ChatGPT was the friend he confided in

#240

Post by stavros »

sn0wleppard wrote: post #235
stavros wrote: post #229

Earlier quoted context omitted.

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

Nice place to cut the quote there > [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

Ah, I misread that and thought that's what the user said.
Post reply on HN