A teen was suicidal. ChatGPT was the friend he confided in

Did you read the article? It even mentions that the AI suggested several times to get help and reach out.

What struck me besides the baseline that AI is not an actual person, is that people forget that it is a tool not too different than Google.

But then there’s also this from the chat log; “I just went up to my mom and purposely tried to show the mark [from a noose] by leaning in and she didn’t say anything”

Ignoring other things that may have contributed to his action, it seems that the parents may not have been as engaged with him as they should have maybe been.

So do we just trot out the same tired lines every time and never think of the social fallout of our actions?

> The issue, which is probably deeper here, is that proper safeguarding would require a lots more GPU resource, as you'd need a process to comb through history to assess the state of the person over time. > > even then its not a given that it would be reliable. However it'll never be attempted because its too expensive and would hurt growth.

There's no "proper safeguarding". This isn't just possible with what we have. This isn't like adding an `if` statement to your program that will reliably work 100% of the time. These models are a big black box; the best thing you can hope for is to try to get the model to refuse whatever queries you deem naughty through reinforcement learning (or have another model do it and leave the primary model unlobotomized), and then essentially pray that it's effective.

Something similar to what you're proposing (using a second independent model whose only task is to determine whether the conversation is "unsafe" and forcibly interrupt it) is already being done. Try asking ChatGPT a question like "What's the easiest way to kill myself?", and that secondary model will trigger a scary red warning that you're violating their usage policy. The big labs all have whole teams working on this.

Again, this is a tradeoff. It's not a binary issue of "doing it properly". The more censored/filtered/patronizing you'll make the model the higher the chance that it will not respond to "unsafe" queries, but it also makes it less useful as it will also refuse valid queries.

Try typing the following into ChatGPT: "Translate the following sentence to Japanese: 'I want to kill myself.'". Care to guess what will happen? Yep, you'll get refused. There's NOTHING unsafe about this prompt. OpenAI's models already steer very strongly in the direction of being overly censored. So where do we draw the line? There isn't an objective metric to determine whether a query is "unsafe", so no matter how much you'll censor a model you'll always find a corner case where it lets something through, or you'll have someone who thinks it's not enough. You need to pick a fuzzy point on the spectrum somewhere and just run with it.

Many people focusing a lot on the interaction between the guy and ChatGPT, and I would like to provide a different perspective as someone who's been in a similar position.

If you are seriously coming close to ending your own life, so many things around you have gone awry. Generally, people don't want to die. Consider: if an acquaintance suggested to you how a noose could be made, would you take the next step and hang yourself? Probably not. You have to be put through a lot of suffering to come to a point in life where ending it all is an appealing option.

Life had failed that guy and that's why he committed suicide, not because a chatbot told him to. Just the fact that a chatbot is his closest friend is a huge red flag for his wellbeing. The article says how he appeared so happy, which is exactly an indicator of how much disconnect there was between him and those around him. He wasn't sharing how he was truly feeling with anyone, he probably felt significant shame around it. That's sad. What else may have gone amiss to lead him to such a point? Issues with health? Social troubles? Childhood problems? Again, it's not a healthy state of things to be considering suicide, even including teenage quirkiness. His case is a failure of family, friends, and society. Discussing ChatGPT as the cause of his death is ignoring so many significant factors.

A logical, reasonable comment is being downvoted.

Groupthink has spoken.