Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

261–270 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#261

Post by Al-Khwarizmi »

bayindirh wrote: post #249
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

Maybe it's causing even more deaths than we know, and these doesn't make the news either? If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be sur…

> Maybe it's causing even more deaths than we know, and these doesn't make the news either?

Of course, and that's part of why I say that we need to measure the impact. It could be net positive or negative, we won't know if we don't find out.

> If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be surprised if these actually transports some people with comfort, and that's not going to make the news.

I'm not advocating for not improving security, I'm arguing against a comment that said that "ChatGPT should be nowhere near anyone dealing with psychological issues", because it can cause death.

Following your analogy, cars objectively cause deaths (and not only of people with psychological issues, but of people in general) and we don't say that "they should be nowhere near a person". We improve their safety even though zero deaths is probably impossible, which we accept because they are useful. This is a big-picture approach.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#262

Post by Kevin605 »

bondarchuk wrote: post #252

It becomes a problem when people cannot distinguish real from fake. As long as people realize they are talking to a piece of software and not a real person, "suicidal people shouldn't be allowed to use LLMs" is almost on par with "suicidal people shouldn't be allowed to read books", or "operate a dvd player", or "listen to alt-rock from the 90s". The real problem is of course grossly deficient mental health care and…

I don’t buy the “LLMs = books” analogy. Books are static; today’s LLMs are adaptive persuasion engines trained to keep you engaged and to mirror your feelings. That’s functionally closer to a specialized book written for you, in your voice, to move you toward a particular outcome. If there exists a book intended to persuade its readers into committing suicide, it would surely be seen as dangerous for depressed people.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#263

Post by wongarsu »

grey-area wrote: post #253
Al-Khwarizmi wrote: post #247

Earlier quoted context omitted.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

First, do no harm.

That relates more to purposefully harming some people to safe other people. Doing something that has the potential to harm a person but statistically has a greater likelihood of helping them is something doctors do all the time. They will even use methods that are guaranteed to do harm to the patient, as long as they have a sufficient chance to also bring a major benefit to the same patient

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#264

Post by probably_wrong »

ben_w wrote: post #254
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No. By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much . Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense". Now this part: > Add in the commercial incentives of 'O…

While I agree with most of your comment, I'd like to dispute the story about GPT-2.

Yes, they did claim that they wouldn't release GPT-2 due to unforeseen risks, but...

a. they did end up releasing it,

b. they explicitly stated that they wouldn't release GPT-3[1] for marketing/financial reasons, and

c. it being dangerous didn't stop them from offering the service for a profit.

I think the quotes around "open" are well deserved.

[1] Edit: it was GPT-4, not GPT-3.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#265

Post by jrflowers »

Al-Khwarizmi wrote: post #236
probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

[flagged]

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#266

Post by pzlarsson »

Al-Khwarizmi wrote: post #247
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

When evaluating good vs harm for drugs or other treatments the risk for lethal side effects must be very small for the treatment to be approved. In this case it is also difficult to get reliable data on how much good and harm is done.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#267

Post by jrflowers »

NathanKP wrote: post #3

> It is estimated that more than one in five U.S. adults live with a mental illness (59.3 million in 2022; 23.1% of the U.S. adult population). https://www.nimh.nih.gov/health/statistics/mental-illness Most people don't understand just how mentally unwell the US population is. Of course there are one million talking to ChatGPT about suicide weekly. This is not a surprising stat at all. It's just a question of what to…

> At least OpenAI is trying to do something about it.

They can certainly say that their chat bot has a documented history of attempting to reduce the number of suicidal people.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#268

Post by noneparticular »

true_religion wrote: post #194
koakuma-chan wrote: post #179

Earlier quoted context omitted.

> and start meeting people in real life Do you have real life hobbies or something? I don't understand how this is supposed to work. I only ever go outside for groceries or gym, etc.

I'm not going to say it's 'simple' to have hobbies or find people, but realistically if you don't regularly meet strangers in real life, you'll never date strangers so it's a catch 22. Unless we all want to set ourselves up for arranged marriages in the future, we need to confront this reality.

Speaking as a pariah for most of his life; I doubts it would ever be so dire.

There's always going to be social circles and people coupling up no matter what. But if anything I wonder if, for people like me who aren't really worthy of intimacy, living in a society has options to live a solitary life while still contributing is actually a net positive overall. For me to self select out of the dating pool would mean less noise for someone else looking for a worthy partner.

There's less chaff that people in said said pool would have to wade though. The people that want to couple and are capable of doing so will continue to so with less distraction. That seems an overall good thing, no?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#269

Post by arowthway »

probably_wrong wrote: post #258
torben-friis wrote: post #248

Earlier quoted context omitted.

>should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Is that a debate worth having though? If the tool is available universally it is hard to imagine any way to stop access without extreme privacy measures. Blocklisting people would require public knowledge of their issues, and one risks the law enforcement effect, where people don’t seek help for fear that it ends up in…

> Is that a debate worth having though? Yes. Otherwise we're accepting "OpenAI wants to do this so we should quietly get out of the way". If ChatGPT has "PhD-level intelligence" [1] then identifying people using ChatGPT for therapy should be straightforward, more so users with explicit suicidal intentions. As for what to do, here's a simple suggestion: make it a three-strikes system. "We detected you're using ChatGPT…

Are people using ChatGPT for therapy more vulnerable than people using it for medical or legal advice? From my experience talking about your problems to the unaccountable bullshit machine is not very different then the "real" therapy.
Post reply on HN