Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

301–310 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#301

Post by david-gpu »

jrflowers wrote: post #265
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

[flagged]

They didn't say it was impossible, or that we should do nothing. Learn how to have a constructive dialogue, please.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#302

Post by lazide »

grey-area wrote: post #245
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Bleach should also not be allowed near suicidal teens.

But how do you tell before it matters?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#303

Post by wafflemaker »

imiric wrote: post #284
bayindirh wrote: post #251

Earlier quoted context omitted.

> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full…

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can…

There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen.

>AI companies are partly responsible for promoting these tools as something more than statistical models,[...]

This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM based AI shortcomings that result from their architecture and are bound to happen.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#304

Post by p1dda »

probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

Exactly, any company that offers chatbots to the public should do what Google did regarding suicide searches, remove harmful websites and provide info how to contact mental health professionals. Anything else would be corporate suicide (pun not intended).

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#305

Post by b3lvedere »

probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

"One may think that something is better than nothing, but a bot enabling your destructive impulses is indeed worse than nothing."

And how would a layman know the difference?

If i desperately need help with mental item x and i have no clue how to get help, am very very ashamed for even mentioning to ask for help about mental item x or there are actually no resources available, i will turn to anything else than nothing. Because item x still exists and is making me suffer 24/7.

At least the bot pretends to listen, some humans cannot even do that.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#306

Post by wafflemaker »

imiric wrote: post #284
bayindirh wrote: post #251

Earlier quoted context omitted.

> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full…

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can…

There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen.

>AI companies are partly responsible for promoting these tools as something more than statistical models,[...]

This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM-based-AI shortcomings that result from their architecture and are bound to happen.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#307

Post by johnisgood »

cmsj wrote: post #294
imiric wrote: post #284

Earlier quoted context omitted.

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can…

> The best they can do is raise a flag Depending on where you live, this may well result in the vulnerable person being placed under professional supervision that actively prevents them from dying. That's a fair bit more valuable than when you describe it as raising a flag.

Yeah... I have been in a locked psychiatric ward many times before and never in my life I came out better. They only address the physical part there for a few days and kick you out until next time. Or do you think people should be physically restrained for a long time without any actual help?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#308

Post by torben-friis »

probably_wrong wrote: post #258
torben-friis wrote: post #248

Earlier quoted context omitted.

>should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Is that a debate worth having though? If the tool is available universally it is hard to imagine any way to stop access without extreme privacy measures. Blocklisting people would require public knowledge of their issues, and one risks the law enforcement effect, where people don’t seek help for fear that it ends up in…

> Is that a debate worth having though? Yes. Otherwise we're accepting "OpenAI wants to do this so we should quietly get out of the way". If ChatGPT has "PhD-level intelligence" [1] then identifying people using ChatGPT for therapy should be straightforward, more so users with explicit suicidal intentions. As for what to do, here's a simple suggestion: make it a three-strikes system. "We detected you're using ChatGPT…

>Yes. Otherwise we're accepting "OpenAI wants to do this so we should quietly get out of the way".

I think it’s fair to demand that they label/warn about the intended usage, but policing it is distopic. Do car manufacturers immediately call the police when the speed limit is surpassed? Should phone manufacturers stop calls when the conversation deals with illegal topics?

I’d much rather regulation went the exact opposite way, seriously limiting the amount of analysis they can run over conversations, particularly when content is not deanonimised.

If there’s something we don’t want is OpenAI storing data about mental issues and potentially selling it to insurers for example. The fact that they could be doing this right now is IMO much more dangerous than tool misuse.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#309

Post by SiempreViernes »

Al-Khwarizmi wrote: post #278
latexr wrote: post #275

Earlier quoted context omitted.

> Supposing that the advice it provides does more good than harm That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument. I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the proble…

In my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.

> we should measure whether in fact it does more good than harm or not

The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.

Post reply on HN