OpenAI says over a million people talk to ChatGPT about suicide weekly • Hacker News Board

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#481

Post by terminalshort » Tue, Oct 28, 2025, 2:57 PM UTC

alex-moon wrote: post #317

Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good en…

This is nothing but an appeal to authority and fear of the unknown. The article linked isn't even able to make a statement stronger than speculation like "may not only lack effectiveness" and "could also contribute to harmful stigma and dangerous responses."

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#482

Post by floor2 » Tue, Oct 28, 2025, 2:57 PM UTC

basisword wrote: post #374

formerly_proven wrote: post #351

Earlier quoted context omitted.

> There are 24/7 suicide prevention hotlines at least in many countries in Europe as well as US states. My understanding is these will generally just send the cops after you if the operator concludes you are actually suicidal and not just looking for someone to talk to for free.

I mean that's clearly a good thing. If you are actually suicidal then you need someone to intervene. But there is a large gulf between depressed and suicidal and those phone lines can help without outside assistance in those cases.

> just send the cops after you

> > that's clearly a good thing

You might want to read up on how interactions between police and various groups in the US tend to go. Sending the cops after someone is always going to be dangerous and often harmful.

If the suicidal person is female, white and sitting in a nice house in the suburbs, they'll likely survive with just a slightly traumatizing experience.

If the suicidal person is male, black or has any appearance of being lower class, the police are likely to treat them as a threat, and they're more likely to be assaulted, arrested, harassed or killed than they are to receive helpful medical treatment.

If I'm ever in a near-suicidal state, I hope no one calls the cops on me, that's a worst nightmare situation.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#483

Post by terminalshort » Tue, Oct 28, 2025, 2:59 PM UTC

alex-moon wrote: post #396

ethbr1 wrote: post #333

Earlier quoted context omitted.

You're holding up a perfect status quo that doesn't correspond to reality. Countries vary, but in the US and many places there's a shortage of quality therapists. Thus for many people the actual options are {no therapy} and {LLM therapy}. > This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed t…

I don't accept the unregulated and uncontrolled use of LLMs for therapy for the same reason I don't accept arguments like "We should deregulate food safety because it means more food at lower cost to consumers" or "We should waive minimum space standards for permitted developments on existing buildings because it means more housing." We could "solve" the homeless problem tomorrow simply by building tenements (that is…

I bet you don't accept that because you can afford the expensive regulated version.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#484

Post by terminalshort » Tue, Oct 28, 2025, 3:00 PM UTC

spookie wrote: post #353

ethbr1 wrote: post #333

Earlier quoted context omitted.

You're holding up a perfect status quo that doesn't correspond to reality. Countries vary, but in the US and many places there's a shortage of quality therapists. Thus for many people the actual options are {no therapy} and {LLM therapy}. > This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed t…

Ok then, the LLMs must pass the same tests and be as regulated as therapists. After all, it should be easy peasy (:

LLMs can pass the bar now, so I don't think they would have any problems here.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#485

Post by wfleming » Tue, Oct 28, 2025, 3:01 PM UTC

Al-Khwarizmi wrote: post #236

probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

If a therapist helped 99/100 patients but tacitly encouraged the 100th to commit suicide* they would still lose their license.

* ignoring the case of ethical assisted suicide for reasons of terminal illness and such, which doesn’t seem relevant to the case discussed here.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#486

Post by prewett » Tue, Oct 28, 2025, 3:01 PM UTC

danaris wrote: post #342

mschuster91 wrote: post #335

Earlier quoted context omitted.

> If the client needs to be reminded to box breathe and that they're using all or nothing thinking again to get them off of the ledge, does that really require a human who's only available once a week to gently remind them of that when the therapist isn't going to be available for four more days and ChatGPT's available right now? There are 24/7 suicide prevention hotlines at least in many countries in Europe as well…

And the reason for this brokenness is all too easy to identify: the very wealthy have been increasingly siphoning off all gains in productivity since the Reagan era. Tax the rich massively, use the money to provide for everyone, without question or discrimination, and most of these issues will start to subside. Continue to wail about how this is impossible, there's no way to make the rich pay their fair share (or, wo…

I guess if all you have is a hammer...

It's certainly easy to blame the rich for everything, but the rich have a tendency to be miserable (the characters in "The Great Gatsby" and "Catcher in the Rye" are illustrations of this). Historically, poor places have often been happier, because of a rich web of social connection, while the rich are isolated and unhappy. [1] Money doesn't buy happiness or psychological well-being, it buys comfort.

A more trenchant analysis of the mental health problem is that the US has designed ourselves into isolation, and then the Covid lockdowns killed a lot of what was left. People need to be known and loved, and have people to love and care about, which obviously cannot happen in isolation.

[1] I am NOT saying that poor = happy, and I think the positive observations tended to be in poor countries, not tenements in London.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#487

Post by xpe » Tue, Oct 28, 2025, 3:01 PM UTC

ThrowawayTestr wrote: post #399

willis936 wrote: post #349

Earlier quoted context omitted.

It's a matter of trust and incentives. How can you trust a program curated by an entity with no accountability? A therapist has a personal stake in helping patients. An LLM provider does not. Seeking help should not be so taboo as people are resorting to doing it alone at night while no one is looking. That is society loudly saying "if you slip off the golden path even a little your life is over". So many people reso…

How can I trust a therapist that has a financial incentive to keep me seeing them?

I'll start with a direct response, because otherwise I suspect my answer may come across as too ... complex.

> How can I trust a therapist that has a financial incentive to keep me seeing them?

The direct response: I hope the commenter isn't fixated on this framing of the question, because I don't think it is a useful framing. [1] What is a better framing, then? I'm not going to give a simple answer. My answer is more like a process.

I suggest refining one's notion of trust to be "I trust Person A to do {X, Y, Z} because of what I know about them (their incentives, professional training, culture, etc)."

Shift one's focus and instead ask: "What aspects of my therapist are positives and/or lead me to trust their advice? What aspects are negative and/or lead me to not trust their advice?" Put this in writing and put some time into it.

One might also want to journal on "How will I know if therapy is helping? What are my goals?" By focusing on this, I think answers relating to "How much is my therapist helping?" will become easier to figure out.

[1] I think it is not useful because both because it is loaded and because it is overly specific. Instead, focus on figuring out what actions one should take. From here, the various factors can slot in naturally.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#488

Post by terminalshort » Tue, Oct 28, 2025, 3:03 PM UTC

grey-area wrote: post #245

Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

> A word generator with no intelligence or understanding

I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#489

Post by buellerbueller » Tue, Oct 28, 2025, 3:04 PM UTC

weatherlite wrote: post #180

diamond559 wrote: post #138

Earlier quoted context omitted.

They're doing something about it alright, they're monetizing their pain for shareholder gainz!

Sure but your therapist is also monetizing your pain for his own gain. Either A.I therapy works (e.g can provide good mental relief) or it doesn't. I tend to think it's gonna be amazing at those things talking from experience (very rough week with my mom's health deterioriating fast, did a couple of sessions with Gemini that felt like I'm talking to a therapist). Perhaps it won't work well for hard issues like real m…

Depression, ADHA, Schizophrenia are not a "very rough week"

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#490

Post by terminalshort » Tue, Oct 28, 2025, 3:06 PM UTC

toofy wrote: post #325

Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

people are overcomplicating this, the big picture is simple af: if a therapist was ever found to have said to this to a suicidal person, they would be immediately stripped of their license and maybe jailed.

But you would have to be a therapist. If a suicidal person went up to a stranger and started a conversation, there would be no consequences. That's more analogous to ChatGPT.