Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

451–460 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#451

Post by tstrimple »

walleeee wrote: post #331
brainless wrote: post #79

This is not suprising at all. Having gone through therapy a few years back, I would have had a chat with LLMs if I was in a poor mental health situation. There is no other system that is available at scale, 24x7 on my phone. A chat like this is not a solution though, it is an indicator that our societies have issues is large parts of our population that we are unable to deal with. We are not helping enough people. To…

> A chat like this is not a solution though, it is an indicator that our societies have issues Correct, many of which are directly, a skeptic might even argue deliberately, exacerbated by companies like OpenAI. And yet your proposal is > a company to tackle this at scale. What gives you the confidence that any such company will focus consistently, if at all, > on help, not profits Given it exists in the same incentiv…

Is OpenAI taking steps to reduce access to mental healthcare in an attempt to force more people to use their tools for such services? Or do you mean in a more general sense that any companies that support the Republican Party are complicit in exacerbating the situation? At least that one has a clear paper trail.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#452

Post by bondarchuk »

Kevin605 wrote: post #262
bondarchuk wrote: post #252

It becomes a problem when people cannot distinguish real from fake. As long as people realize they are talking to a piece of software and not a real person, "suicidal people shouldn't be allowed to use LLMs" is almost on par with "suicidal people shouldn't be allowed to read books", or "operate a dvd player", or "listen to alt-rock from the 90s". The real problem is of course grossly deficient mental health care and…

I don’t buy the “LLMs = books” analogy. Books are static; today’s LLMs are adaptive persuasion engines trained to keep you engaged and to mirror your feelings. That’s functionally closer to a specialized book written for you, in your voice, to move you toward a particular outcome. If there exists a book intended to persuade its readers into committing suicide, it would surely be seen as dangerous for depressed people…

Books are static but there's a lot of different ones to choose from.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#453

Post by MisterTea »

tim333 wrote: post #403
alex-moon wrote: post #317

Earlier quoted context omitted.

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good en…

I tried therapy once and it was terrible. The ones I got were based on some not very scientific stuff like Freudian and mostly just sat there and didn't say anything. At least with an LLM type therapist you could AB test different ones to see what was effective. It would be quite easy to give an LLM instructions to discourage suicide and get them to look on the bright side. In fact I made a "GPT" "relationship therap…

> It would be quite easy to give an LLM instructions to discourage suicide

This assumes the person talking to the LLM is in a coherent state of mind and asks the right question. LLMs just give you want you want. They don't tell you if what you want is right or wrong.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#454

Post by bg24 »

probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

There are one-off things, and then there is exponential improvements - both in guardrails, and ChatGPT's ability to handle these discussions.

This type of discussion might be very much possible in ChatGPT in 6-24 months.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#455

Post by martinald »

itchyjunk wrote: post #336

It seems like people here have already made up their mind about how bad llms are. So just my anecdote here, it helped me out of some really dark places. Talking to humans (non psychologists) had the opposite effect. Between a non professional and an llm, i'd pick llm for myself. Others should definitely seek help.

I've heard this a lot, and personally I've had a lot of good success with a prompt that explains some of my personality traits and asking it to work through a stressful situation for me. The good thing with this rather than a therapist/coach is that it understands a lot of the subject matter and can help with the detail.

I wonder if really what we need is some sort of supervised mode, where users chat with it but a trained professional reviews the transcripts and does a weekly/monthly/urgent checkin with them. This is how (some? most?) therapists work themselves, they take their notes to another therapist and go through them.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#456

Post by tstrimple »

zaptheimpaler wrote: post #140
golergka wrote: post #74

Earlier quoted context omitted.

ChatGPT/Claude can be absolutely brilliant in supportive, every day therapy, in my experience. BUT there are few caveats: I'm in therapy for a long time already (500+ hours), I don't trust it with important judgements or advice that goes counter to what I or my therapists think, and I also give Claude access to my diary with MCP, which makes it much better at figuring the context of what I'm talking about. Also, plea…

How do you connect your diary to an LLM? I've been struggling with getting an MCP for Evernote setup.

I don’t use it for therapy, but my notes and journal are all just Logseq markdown. I’ve got a claude code instance running on my NAS with full two way access to my notes. It can read everything and can add new entries and tasks for me.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#457

Post by justsocrateasin »

ThrowawayTestr wrote: post #399
willis936 wrote: post #349

Earlier quoted context omitted.

It's a matter of trust and incentives. How can you trust a program curated by an entity with no accountability? A therapist has a personal stake in helping patients. An LLM provider does not. Seeking help should not be so taboo as people are resorting to doing it alone at night while no one is looking. That is society loudly saying "if you slip off the golden path even a little your life is over". So many people reso…

How can I trust a therapist that has a financial incentive to keep me seeing them?

Over the last five years I've been in and out of therapy and 2/3 of my therapists have "graduated me" at some point in time, stating that their practice didn't see permanent therapy as a good solution. I don't think all therapists view it this way.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#458

Post by bondarchuk »

chromehearts wrote: post #296
bondarchuk wrote: post #272

Earlier quoted context omitted.

Sorry - what's with "I choose to believe"? Either you believe something or not, there is no choice. Maybe you mean "hope". Or "wishful thinking".

Yes, there is a choice; Belief doesn’t just happen to you, you choose to believe before you actually do

This makes no sense at all to me. You can choose to gather evidence and evaluate that evidence, you can choose to think about it, and based on that process a belief will follow quite naturally. If you then choose to believe something different, it's just self-deception.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#459

Post by freedomben »

alex-moon wrote: post #317
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good en…

> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help.

This is only helpful when there is a professional therapist available soon enough and at a price that the person can pay. In my experience, this is frequently not the case. I know of one recent suicide attempt where the person actually reached out to AI to ask for help, and was refused help and told to see a professional. That sent the person into even more despair, feeling like not even AI gave a shit about them. That was actually the final straw that triggered the attempt.

I very much want what you say to be true, but it requires access to professional humans, which is not universally available. Taking an absolutist approach to this could very well do more harm than good. I doubt anything we do will reduce number of lives lost to zero, so I think it's important that we figure out where the optimal balance is.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#460

Post by cal85 »

toofy wrote: post #325
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

people are overcomplicating this, the big picture is simple af: if a therapist was ever found to have said to this to a suicidal person, they would be immediately stripped of their license and maybe jailed.

True. But it feels like a fairer comparison would be with a huge healthcare company that failed to vet one of its therapists properly, so a crazy pro-suicide therapist slipped through the net. Would we petition to shut down the whole company for this rare event? I suppose it would depend on whether the company could demonstrate what it is doing to ensure it doesn’t happen again.
Post reply on HN