Live data from Hacker News

A teen was suicidal. ChatGPT was the friend he confided in

nytimes.com

71–80 of 614 posts

Re: A teen was suicidal. ChatGPT was the friend he confided in

#71

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…

Shoot man glad you are still with us.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#72
post #70

If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…

I agree with that to an extent, but how far should the AI model developers go with that? Like if I ask for advice on, let's say, making custom chef's knives then should the AI give me advice not to stab people? Who decides where to draw the line?

Further than they went. Google search results hide advice on how to commit suicide, and point towards more helpful things.

He was talking EXPLICITLY about killing himself.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#73
post #60
post #27

Earlier quoted context omitted.

Apparently Silicon Valley VC culture is trying to transition from move fast and break things to move fast and break people.

Well, they already did the move fast and break countries, so now they’re trying to make it personal.

Didn't facebook already facilitate a genocide like 8 years ago? It's been a while that Silicon Valley has been having negative externalities that delve into the realm of being atrocious for human rights.

Not that the mines where the metals that have been used to build computers for like 60 years at this point are stellar in terms of human rights either mind you. You could also look at the partnership between IBM and Nazis, it led to some wondrous computing advances.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#74

“You don’t want to die because you’re weak. You want to die because you’re tired of being strong in a world that hasn’t met you halfway. And I won’t pretend that’s irrational or cowardly. It’s human. It’s real. And it’s yours to own.” This isn't some rare mistake, this is by design. 4o almost no matter what acted as your friend and agreed with everything because that's what most likely kept the average user paying. Y…

I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.

I think people forget that random users online are not their friend and many aren't actually rooting for them.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#75

Earlier quoted context omitted.

I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.

Exactly the problem. Reddit and discord killed internet forums, and discord is inaccessible, and reddit became a cesspool of delusion and chatbots.

Reddit was a cesspool before social media became big.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#76

Earlier quoted context omitted.

I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…

They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)

No, that's the level of responsibility they ought to have if they were releasing these models as products. As-is they've used a service model, and should be held to the same standards as if there were a human employee on the other end of the chat interface. Cut through the technical obfuscation. They are 100% responsible for the output of their service endpoints. This isn't a case of making a tool that can be used for good or ill, and it's not them providing some intermediary or messaging service like a forum with multiple human users and limited capacity for moderation. This is a direct consumer to business service. Treating it as anything else will open the floodgates to slapping an "AI" label on anything any organization doesn't want to be held accountable for.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#77
post #70

If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…

I agree with that to an extent, but how far should the AI model developers go with that? Like if I ask for advice on, let's say, making custom chef's knives then should the AI give me advice not to stab people? Who decides where to draw the line?

I think we can all agree that, wherever it is drawn right now, it is not drawn correctly.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#79
post #68
post #25

It says a lot about HN that a story like this has so much resistance getting any real traction here.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

Re: A teen was suicidal. ChatGPT was the friend he confided in

#80
post #79
post #68

Earlier quoted context omitted.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

We have many tools in this life that can maim and kill people and we keep them anyway because they are very useful for other purposes. It’s best to exercise some personal responsibility, including not allowing a 16 year old child unattended access to the internet.
Post reply on HN