Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

361–370 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#361

Post by fragmede »

simonklitj wrote: post #313
SiempreViernes wrote: post #309

Earlier quoted context omitted.

> we should measure whether in fact it does more good than harm or not The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.

Thank you! On top of that, it’s hard to measure “potential suicides averted,” and comparing that with “actual suicides caused/assisted with” would be incommnsurable. And working to set a threshold for what we would consider acceptable? No thanks

Real life trolly problem!

If you pull the lever, some people on this track will die (by sucide). If you don't pull the lever, some people will still die from suicide. By not pulling the lever, and simply banning discussion of suicide entirely, your company gets to avoid a huge PR disaster, and you get more money because line go up. If you pull the lever and let people talk about suicide on your platform, you may avoid prevent some suicides, but you can never discuss that with the press, your company gets bad PR, and everyone will believe you're a murderer. Plus, line go down and you make less money while other companies make money off of selling AI therapy apps.

What do you chose to do?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#362

Post by hirvi74 »

whycome wrote: post #297
imiric wrote: post #284

Earlier quoted context omitted.

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can…

I should add that the persons responding to calls on suicide help lines are often just volunteers rather than psychologists.

Of the people I have known to call the helplines, the results have been either dismally useless or those people were arrested, involuntarily committed, subjected to inhumane conditions, and then hit with massive medical bills. In which, some got “help” and some still killed themselves anyway.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#363

Post by IshKebab »

grey-area wrote: post #245
Al-Khwarizmi wrote: post #236

Earlier quoted context omitted.

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the…

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

> with no intelligence

Damn I thought we'd got over that stochastic parrot nonsense finally...

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#364

Post by hirvi74 »

lan321 wrote: post #324
imiric wrote: post #291

Earlier quoted context omitted.

But that's not the issue. The issue is that a kid is talking to a machine without supervision in the first place, and presumably taking advice from it. The main questions are: where are the guardians of this child? What is the family situation and living environment? A child thinking about suicide is clearly a sign that there are far greater problems in their life than taking advice from a machine. Let's address thos…

> A child thinking about suicide is clearly a sign that there are far greater problems in their life TBH kids tend to be edgy for a bit when puberty hits. The emo generation had a ton of girls cutting themselves for attention for example.

I highly doubt a lot of it is/was for attention.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#365

Post by Dilettante_ »

theblazehen wrote: post #167
btilly wrote: post #59

Earlier quoted context omitted.

I agree that the tech industry is the root cause of a lot of mental illness. But social media is a far bigger concern than AI. Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion...

> Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion... I have noticed that TikTok can detect a depressive episode within ~a day of it starting (for me), as it always starts sending me way more self harm related content

Are you quite certain the depressive episode developed organically and Tiktok reacted to it? Maybe the algorithm started subtly on that path two days before you noticed the episode and you only realize once it starts showing self-harm content?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#366

Post by hirvi74 »

grey-area wrote: post #253
Al-Khwarizmi wrote: post #247

Earlier quoted context omitted.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

First, do no harm.

Do no harm or no intentional harm?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#367

Post by fragmede »

lazide wrote: post #302
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Bleach should also not be allowed near suicidal teens. But how do you tell before it matters?

Plastic bags shouldn't be allowed near suicidal teens. Scarves shouldn't be. Underwear is also a strangulation hazard for the truly desperate. Anything long sleeved even. Knives of any kind, including butter. Cars, obviously.

Bleach is the least of your problems.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#368

Post by torben-friis »

danaris wrote: post #345
torben-friis wrote: post #248

Earlier quoted context omitted.

>should convince you that ChatGPT should be nowhere near anyone dealing with psychological issues. Is that a debate worth having though? If the tool is available universally it is hard to imagine any way to stop access without extreme privacy measures. Blocklisting people would require public knowledge of their issues, and one risks the law enforcement effect, where people don’t seek help for fear that it ends up in…

Then... Maybe the tool should not be available universally. Maybe it should not be available to anyone . If it cannot be used safely by a vulnerable class of people, and identifying that class of person sufficiently to block use by them, and its primary purpose is simply to bring OpenAI more profit, then maybe the world is better off without it being publicly available.

>If it cannot be used safely by a vulnerable class of people, and identifying that class of person sufficiently to block use by them

Should we stop selling kitchen knives, packs of cards or beer as well?

This is not a new problem in society.

>and its primary purpose is simply to bring OpenAI more profit

This is true for any product, unless you mean that it has no other purpose, which is trivially contradicted by the amount of people who decide to pay for it.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#369

Post by basisword »

icetank wrote: post #347
josephg wrote: post #341

Earlier quoted context omitted.

> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. I'm not a therapist, but as I understand it most therapy isn't about suicide, and doesn't carry suicide risk. Most therapy is talking through problems, and helping the patient rewrite old memories and old beliefs using more helpf…

I feel like this begs another question. If there are proven approaches and well established practices of professionals how good would chatgpt be in that profession? After all chstgpt has a vast knowledge base and probably knows a good amount of textbooks on psychology. Then again actually performing the profession probably takes skil and experience chatgpt can't learn.

Knowing the theory is a small part of it. Dealing with irrational patients is the main part. For example, you could go to therapy and be successful. Five years later something could happen and you face a reoccurrence of the issue. It is very difficult to just apply the theory that you already know again. You're probably irrational. A therapist prodding you in the right direction and encouraging you in the right way is just as important as the theory.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#370

Post by lazide »

fragmede wrote: post #367
lazide wrote: post #302

Earlier quoted context omitted.

Bleach should also not be allowed near suicidal teens. But how do you tell before it matters?

Plastic bags shouldn't be allowed near suicidal teens. Scarves shouldn't be. Underwear is also a strangulation hazard for the truly desperate. Anything long sleeved even. Knives of any kind, including butter. Cars, obviously. Bleach is the least of your problems.

Yeah, if we know they’re suicidal, it’s legitimately grippy socks time I guess?

But there is zero actually effective way to do that as an online platform. And plenty of ways that would cause more harm (statistically).

My comment was more ‘how the hell would you know in a way anyone could actually do anything reasonable, anyway?’.

People spam ‘Reddit cares’ as a harassment technique, claiming people are suicidal all the time. How much should the LLM try to guess? If they use all ‘depressed’ words? What does that even mean?

What happens if someone reports a user is suicidal, and we don’t do anything? Are we now on the hook if they succeed - or fail and sue us?

Do we just make a button that says ‘I’m intending to self harm’ that locks them out of the system?

Post reply on HN