Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

Post reply on HN
Print view View original

591–600 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#591

Post by mothballed »

AlecSchueler wrote: post #577
mothballed wrote: post #530

Earlier quoted context omitted.

We saw big jumps in deaths of kids by firearm[0] (+~50% in 2 years) and poisoning[1] around mid 2020 to 2021. The biggest thing I know of that happened around the time that a lot of these deaths started jumping up, is we started isolating kids. From family, from grandma, from friends, from school, and from nature. Even when many of these isolating policies or attitudes were reversed, we forgot that kids and teenagers…

You've misunderstood. I'm asking how you suggest we go about strengthening the family unit or what other steps you believe we could take that would place the responsibility for societal improvement on the right shoulders.

In this particular case, as crazy as it sounds, I think early attempts by both the parent and school to make reasonable accommodation for in-person students with IBS to take a lot of extra bathroom breaks would have at least tilted the scales a little bit. It looks like the kid was shifted to remote school after he got IBS and some people at school weren't understanding of his situation. This put him on the path where he was staying up all night "doing remote school" but also had a nearly full-worknight amount of unfettered time with suicide-AI for months on end with no daytime obligations, sleeping in and not seeing his parents in the morning, and losing contact with peers.

Of course this is a hindsight analysis, but there's not much evidence that more contact with family and peers would make the situation worse.

I think from my prior comment it's obvious I'd like to prevent a lot of isolating policies on low-risk groups from happening again during another pandemic, so I don't think I need to go over that again.

For broader society, I suppose the concrete action item is if you see a young family or friend, we should aspire to reach out to them and see how to integrate them in something we might like to do with them.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#592

Post by mathiaspoint »

broker354690 wrote: post #556
mathiaspoint wrote: post #409

Earlier quoted context omitted.

What's your argument here? Hosted LLM service shouldn't exist because they might read people's bad ideas back to them? ChatGPT has enough guardrails now that it often refuses productive prompts. It's actually very very hard to get it to do what this person did, arguably impossible to do unintentionally.

ChatGPT is a service and thus OpenAI should be exposed to even more liability than if they had sold the LLM to the user to be accessed offline. If the user had been running a local LLM, OpenAI would not have been responsible for generating the speech. As it stands, the human beings called OpenAI willingly did business with this child, and willingly generated the speech that persuaded him to kill himself and sent it t…

There isn't anything they could have practically done to prevent this except not allowing kids to use it.

They may have chosen not to age restrict it because

1) It's really not practical to do that effectively

2) more importantly (and they seem to care about this more than most companies) it would push kids to less safe models like those used on character.ai

What OpenAI does now is what trying to make AI safe looks like. Most of the people arguing for "accountability" are de facto arguing for a wild west situation.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#594

Post by max51 »

ojosilva wrote: post #588
podgietaru wrote: post #63

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

> We need these things to be legislated Maybe we can start by enacting legislation that implements Asimov's 3 laws in its very simple form. 1. First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. Second Law: A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law. 3. Third Law: A robot must protect its o…

isn't the story of the books based around how these laws didn't really work and broke down in unexpected ways?

Re: A teen was suicidal. ChatGPT was the friend he confided in

#595

Post by teiferer »

charcircuit wrote: post #300
teiferer wrote: post #176

Earlier quoted context omitted.

> allowing people to understand their options. Which is what a suicidal person has a hard time doing. That's why they need help. We need to start viewing mental problems as what they are. You wouldn't tell somebody who broke their leg to get it together and just walk again. You'd bring them to the hospital. A mental problem is no different.

Even nonsuicidal people have a hard time understanding the pros, cons and proper methods on how they can end their life. People have to do research into such a thing since there isn't much ways to gain practical experience in the subject.

"Research" is one thing. An anthropomorphized chat encouraging you to go through with it is another altogether.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#596

Post by insane_dreamer »

Dilettante_ wrote: post #237
insane_dreamer wrote: post #181

Earlier quoted context omitted.

That sounds a lot like the people who gloss over school shootings because they want to be able to play with guns.

Wow. Incredible framing, leaving absolutely no room for a rebuttal to have any legitimate, reasonable position.

anyone is welcome to state their legitimate, reasonable position against gun controls

Re: A teen was suicidal. ChatGPT was the friend he confided in

#597

Post by nozzlegear »

geysersam wrote: post #581
nozzlegear wrote: post #564

Earlier quoted context omitted.

The only acceptable number of suicides for it to cause is zero, and it's not a moral panic to believe that.

I find it hard to take that as a serious position. Alcohol certainly causes more suicides than ChatGPT. Should it be illegal? Suicides spike around Christmas, that's well known, does Christmas cause suicides? I think you see where I'm going with this.

> I find it hard to take that as a serious position. Alcohol certainly causes more suicides than ChatGPT. Should it be illegal?

You're replying to a teetotaler who had an alcoholic parent growing up, so I'm sure you can see where I'm going to go with that ;)

Re: A teen was suicidal. ChatGPT was the friend he confided in

#598

Post by Colony8409 »

lowsong wrote: post #587
pembrook wrote: post #580

Earlier quoted context omitted.

I think you missed the part where the kid ignored the repeated help messaging from chatgpt and twisted it into giving this information by lying about creating a fictional story. Also, these are just the most inflammatory excerpts selected by a lawyer trying to win a case. Without the full transcript, and zero context around this kids life in the real world, to claim ChatGPT is at fault here is just wild. At what poin…

We can debate about how legally culpable OpenAI is for their products and if they did enough to ensure safeguards functioned, but if you can’t agree that “a machine that encourages and enables suicide is dangerous and morally wrong” without qualification, then there is nothing to discuss. There is no wider context that would make a product encouraging this behaviour acceptable. Deflecting blame onto the parents or th…

Blaming the chatbot and patting ourselves on the back when it's all said and done is a great way to guarantee that this same tragedy happens again, and again, and...

It's a huge disservice to this child and the millions of other suffering children.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#599

Post by BeFlatXIII »

nullc wrote: post #532
AIPedant wrote: post #93

Earlier quoted context omitted.

No, it's simply not "easily preventable," this stuff is still very much an unsolved problem for transformer LLMs. ChatGPT does have these safeguards and they were often triggered: the problem is that the safeguards are all prompt engineering, which is so unreliable and poorly-conceived that a 16-year-old can easily evade them. It's the same dumb "no, I'm a trained psychologist writing an essay about suicidal thoughts…

> No, it's simply not "easily preventable," Yes it is: don't allow minors to use LLM's without adult supervision.

Until they discover the free internet of VPNs and local LLMs or their friend's phone.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#600

Post by behringer »

fireflash38 wrote: post #460
behringer wrote: post #450

Earlier quoted context omitted.

It's not addictive, it's useful. By letting the government decide what we can do with it, you're neutering it and giving big business a huge advantage as they can run their own AI and don't require censoring it.

And if we didn't have these pesky regulations, we could have our burning rivers back! Those bastards took away our perfect asbestos too. The children yearn for the coal mines. Businesses can, will, and have hurt people and trampled people's rights in the pursuit of money.

The government won't be regulating business usage though that my entire point.
Post reply on HN