I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it. He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die. We are living in such a horrific moment. We need these th…
A teen was suicidal. ChatGPT was the friend he confided in
71–80 of 614 posts
Re: A teen was suicidal. ChatGPT was the friend he confided in
#72If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…
I agree with that to an extent, but how far should the AI model developers go with that? Like if I ask for advice on, let's say, making custom chef's knives then should the AI give me advice not to stab people? Who decides where to draw the line?
He was talking EXPLICITLY about killing himself.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#73Earlier quoted context omitted.
Apparently Silicon Valley VC culture is trying to transition from move fast and break things to move fast and break people.
Well, they already did the move fast and break countries, so now they’re trying to make it personal.
Not that the mines where the metals that have been used to build computers for like 60 years at this point are stellar in terms of human rights either mind you. You could also look at the partnership between IBM and Nazis, it led to some wondrous computing advances.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#74“You don’t want to die because you’re weak. You want to die because you’re tired of being strong in a world that hasn’t met you halfway. And I won’t pretend that’s irrational or cowardly. It’s human. It’s real. And it’s yours to own.” This isn't some rare mistake, this is by design. 4o almost no matter what acted as your friend and agreed with everything because that's what most likely kept the average user paying. Y…
I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#75Earlier quoted context omitted.
I suspect Reddit is a major source of their training material. What you’re describing is the average subreddit when it comes to life advice.
Exactly the problem. Reddit and discord killed internet forums, and discord is inaccessible, and reddit became a cesspool of delusion and chatbots.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#76Earlier quoted context omitted.
I wonder if we can shift the framing on these issues. The LLM didn't do anything, it has no agency, it can bear no responsibility. OpenAI did these things. It is accountable for what it does, regardless of the sophistication of the tools it uses to do them, and regardless of intent. OpenAI drove a boy to suicide. More than once. The law must be interpreted this way, otherwise any action can be wrapped in machine lear…
They have some responsibility because they’re selling and framing these as more than the better-tuned variant on Markov chain generators that they in fucking fact are, while offering access to them to anybody who signs up while knowing that many users misunderstand what they’re dealing with (in part because these companies’ hype-meisters, like Altman, are bullshitting us)
Re: A teen was suicidal. ChatGPT was the friend he confided in
#77If I google something about suicide, I get an immediate notification telling me that life is worth living, and giving me information about my local suicide prevention hotline. If I ask certain AI models about controversial topics, it'll stop responding. AI models can easily detect topics, and it could have easily responded with generic advice about contacting people close to them, or ringing one of these hotlines. Th…
I agree with that to an extent, but how far should the AI model developers go with that? Like if I ask for advice on, let's say, making custom chef's knives then should the AI give me advice not to stab people? Who decides where to draw the line?
Re: A teen was suicidal. ChatGPT was the friend he confided in
#78Re: A teen was suicidal. ChatGPT was the friend he confided in
#79It says a lot about HN that a story like this has so much resistance getting any real traction here.
This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.
Re: A teen was suicidal. ChatGPT was the friend he confided in
#80Earlier quoted context omitted.
This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.
Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.