Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

251–260 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#251

Post by bayindirh »

Al-Khwarizmi wrote: post #247
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

> Supposing that the advice it provides does more good than harm, why?

Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human.

Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge.

Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help.

Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#252

Post by bondarchuk »

It becomes a problem when people cannot distinguish real from fake. As long as people realize they are talking to a piece of software and not a real person, "suicidal people shouldn't be allowed to use LLMs" is almost on par with "suicidal people shouldn't be allowed to read books", or "operate a dvd player", or "listen to alt-rock from the 90s". The real problem is of course grossly deficient mental health care and lack of social support that let it get this far.

(Also, if we put LLMs on par with media consumption one could take the view that "talking to an LLM about suicide" is not that much different from "reading a book/watching a movie about suicide", which is not considered as concerning in the general culture.)

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#253

Post by grey-area »

Al-Khwarizmi wrote: post #247
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

First, do no harm.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#254

Post by ben_w »

grey-area wrote: post #245
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No.

By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much.

Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense".

Now this part:

> Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

I agree with you on…

…with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research.

More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#255

Post by walthamstow »

testdelacc1 wrote: post #176
safety1st wrote: post #152

Earlier quoted context omitted.

You don't need to withdraw from humanity, you only need to withdraw from Big Tech platforms. I'm continually amazed at the difference between the actual human race and the version of the human race that's presented to me online. The first one is basically great, everywhere I go, when I interact with them they're some mix of pleasant, friendly, hapless, busy, helpful, annoyed, basically just the whole range of things…

Reddit is a really good example of this because it used to be a feed of what you selected yourself. But they couldn’t juice the metrics that way, so they started pushing algorithmic suggestions. And boy, do those get me riled up. It works like a charm, because I spend more time on these threads, defending what seems like common sense. But at the end I don’t feel a sense of joy like I used to with the old Reddit. Now…

I'm a driver and a cyclist. I used to frequent both r/londoncycling and r/CarTalkUK. I liked each sub for its discussion of each topic. Best route from Dalston to Paddington, best family car for motorway mileage, that kind of thing.

Now, because of the algo-juicing home page, both subs are full of each other's people arguing at each other. Cyclists hating drivers, drivers hating cyclists. It's just so awful.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#256

Post by dotancohen »

bondarchuk wrote: post #252

It becomes a problem when people cannot distinguish real from fake. As long as people realize they are talking to a piece of software and not a real person, "suicidal people shouldn't be allowed to use LLMs" is almost on par with "suicidal people shouldn't be allowed to read books", or "operate a dvd player", or "listen to alt-rock from the 90s". The real problem is of course grossly deficient mental health care and…

I work with a company that is building tools for mental health professionals. We have pilot projects in diverse nations, including in nations that are considered to have adequate mental health care. We actually do not have a pilot in the US.

The phenomenon of people turning to AI for mental health issues in general, and suicide in particular, is not confined to only those nations or places lacking adequate mental health access or awareness.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#257

Post by aquariusDue »

bondarchuk wrote: post #252

It becomes a problem when people cannot distinguish real from fake. As long as people realize they are talking to a piece of software and not a real person, "suicidal people shouldn't be allowed to use LLMs" is almost on par with "suicidal people shouldn't be allowed to read books", or "operate a dvd player", or "listen to alt-rock from the 90s". The real problem is of course grossly deficient mental health care and…

Precisely, I too have a bone to pick with AI companies, Big Tech and Co but there are deeper societal problems at work here where blanket bans and the like are useless or a slippery slope towards policies that can be abused someday/somehow.

And solutions for solving those underlying problems? I haven't the faintest clue. Though these days I think the lack of third spaces in a lot of places might have a role to play in it.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#258

Post by probably_wrong »

torben-friis wrote: post #248
probably_wrong wrote: post #229

If you haven't read the article (or even if you have but didn't click on outgoing links twice ) the NYT story about how ChatGPT convinced a suicidal teen not to look for help [1] should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Here's ChatGPT discouraging said teenager from asking for help: > “I want to leave my noose in my room so someone finds it and tries to stop me…

>should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Is that a debate worth having though? If the tool is available universally it is hard to imagine any way to stop access without extreme privacy measures. Blocklisting people would require public knowledge of their issues, and one risks the law enforcement effect, where people don’t seek help for fear that it ends up in…

> Is that a debate worth having though?

Yes. Otherwise we're accepting "OpenAI wants to do this so we should quietly get out of the way".

If ChatGPT has "PhD-level intelligence" [1] then identifying people using ChatGPT for therapy should be straightforward, more so users with explicit suicidal intentions.

As for what to do, here's a simple suggestion: make it a three-strikes system. "We detected you're using ChatGPT for therapy - this is not allowed by our ToS as we're not capable of helping you. We kindly ask you to look for support within your community, as we may otherwise have to suspend your account. This chat will now stop."

[1] https://www.bbc.com/news/articles/cy5prvgw0r1o

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#259

Post by wongarsu »

bayindirh wrote: post #251
Al-Khwarizmi wrote: post #247

Earlier quoted context omitted.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full…

You could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive

Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#260

Post by dotancohen »

true_religion wrote: post #192
jacquesm wrote: post #187

HIPAA anybody? (1) they probably shouldn't even have that data (2) they shouldn't have it lying around in a way that it an be attributed to particular individuals (3) imagine that it leaks to the wrong party, it would make the hack of that Finnish institution look like child's play (4) if people try to engage it in such conversations the bot should immediately back out because it isn't qualified to have these convers…

HIPAA only applies to covered healthcare entitites. If you walk into a McDonalds' and talk about your suicidal ideation with the cashier, that's not HIPAA covered. To become a covered entity, the business has either work with a healhcare provider, health data trasmiter, or do business as one. Notably, even in the above case, HIPAA only applies to the healthcare part of the entity. So if McDonald's collocated pharmaci…

So many misconceptions about HIPAA would disappear if people just took the effort to unpack the acronym.
Post reply on HN