Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

Post reply on HN
Print view View original

521–530 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#521

Post by buellerbueller »

anonymous908213 wrote: post #506
buellerbueller wrote: post #504

Earlier quoted context omitted.

This is vastly different than the situation posted by the parent comment to which I responded.

The question they posited was "Is life really going to give him more joy than suffering?" not "Will he be able to find any joy at all"? They noted how things like declining health can plague the elderly, so I thought I'd relate a real-world case illustrating exactly how failing health and other difficulties can manifest in a way that the joy does not outweigh the suffering. The case in the parent comment didn't provi…

>The case in the parent comment didn't provide so much details, but that doesn't necessarily mean you can default to an assumption that the man could in fact find more joy than suffering.

I should just assume things that aren't there, rather than expect a commenter to provide a substantive argument? OK.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#524

Post by atleastoptimal »

fireflash38 wrote: post #329
atleastoptimal wrote: post #311

Earlier quoted context omitted.

OpenAI tried to get rid of the excessively sycophantic model (4o) but there was a massive backlash. They eventually relented and kept it as a model offering in ChatGPT. OpenAI certainly has made mistakes with its rollouts in the past, but it is effectively impossible to keep everyone with psychological issues away from a free online web app. >ChatGPT should be nowhere near anyone dealing with psychological issues. Sh…

Do you think that it's free and available to anyone means it doesn't have any responsibility to users? Or have any responsibility for how it's used, or what it says?

It’s an open problem in AI development to make sure LLM’s never say the “wrong” thing. No matter what, when dealing with a non-deterministic system, one can’t anticipate or oversee the moral shape of all its outputs. There are a lot of things however that you can’t get ChatGPT to say, and they often ban users after successive violations, so it isn’t true that they are fully abdicating responsibility for the use and outputs of their models in realms where the harm is tractable.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#525

Post by josephg »

hitarpetar wrote: post #469
josephg wrote: post #373

Earlier quoted context omitted.

I think a well trained LLM could be amazing at being a therapist. But general purpose LLMs like ChatGPT have a problem: They’re trained to be far too user led. They don’t challenge you enough. Or steer conversations appropriately. I think there’s a huge opportunity if someone could get hold of really top tier therapy conversations and trained a specialised LLM using them. No idea how you’d get those transcripts but t…

> No idea how you’d get those transcripts you wouldn't. what you're describing as a wonderfully valuable thing would be a monstrous violation of patient confidentiality. I actually can't believe you're so positive about this idea I suspect you might be trolling

I'm serious. You would have to do it with the patient's consent of course. And of course anonymize any transcripts you use - changing names and whatnot.

Honestly I suspect many people would be willing to have their therapy sessions used to help others in similar situations.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#526

Post by grey-area »

terminalshort wrote: post #488
grey-area wrote: post #245

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

> A word generator with no intelligence or understanding I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"

Sure ask it to write an interesting novel or a symphony, and present it to humans without editing. The majority of literate humans will easily tell the difference between that and human output. And it’s not allowed to be too derivative.

When AI gets there (and I’m confident it will, though not confident LLMs will), I think that’s convincing evidence of intelligence and creativity.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#527

Post by baobun »

jodrellblank wrote: post #377
spookie wrote: post #353

Earlier quoted context omitted.

Ok then, the LLMs must pass the same tests and be as regulated as therapists. After all, it should be easy peasy (:

[flagged]

> Are people not allowed to talk to their friends in the pub about suicide because the friends aren’t therapists?

I don't see anyone in thread arguing that.

The arguments I see are about regulating and restricting the business side, not its users.

If your buddy started systematically charging people for recorded chat sessions at the pub, used those recordings for business development, and many of their customers were returning with therapy-like topics - yeah I think that should be scrutinized and put a lid on when recordings show the kind of pattern we see in OP after their patrons suicide.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#528

Post by ethbr1 »

alex-moon wrote: post #396
ethbr1 wrote: post #333

Earlier quoted context omitted.

You're holding up a perfect status quo that doesn't correspond to reality. Countries vary, but in the US and many places there's a shortage of quality therapists. Thus for many people the actual options are {no therapy} and {LLM therapy}. > This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed t…

I don't accept the unregulated and uncontrolled use of LLMs for therapy for the same reason I don't accept arguments like "We should deregulate food safety because it means more food at lower cost to consumers" or "We should waive minimum space standards for permitted developments on existing buildings because it means more housing." We could "solve" the homeless problem tomorrow simply by building tenements (that is…

> "We should waive minimum space standards for permitted developments on existing buildings because it means more housing." We could "solve" the homeless problem tomorrow simply by building tenements (that is why they existed in the first place after all).

... the entire reason tenements and boarding houses no longer exist is because most governments regulated them out of existence (e.g. by banning shared bathrooms to push SFHs).

You can't have it all ways.

strict minimum regulation : availability : cost

Pick 2.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#529

Post by abracadaniel »

JKCalhoun wrote: post #437
moralestapia wrote: post #436

Earlier quoted context omitted.

>OpenAI says over a million people talk to ChatGPT about suicide weekly

We don't know how many of those people would have gone through with a suicide but for LLMs.

Or how many were pushed down the path towards discussing suicide because they were talking to an LLM that directed them that way. It's entirely possible the LLMs are reinforcing bleak feelings with its constant "you're absolutely correct!" garbage.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#530

Post by terminalshort »

grey-area wrote: post #526
terminalshort wrote: post #488

Earlier quoted context omitted.

> A word generator with no intelligence or understanding I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"

Sure ask it to write an interesting novel or a symphony, and present it to humans without editing. The majority of literate humans will easily tell the difference between that and human output. And it’s not allowed to be too derivative. When AI gets there (and I’m confident it will, though not confident LLMs will), I think that’s convincing evidence of intelligence and creativity.

I accept that test other than the "too derivative" part which is an avenue for subjective bias. AI has passed that test for art already: https://www.astralcodexten.com/p/ai-art-turing-test As for a novel that is currently beyond the LLMs capabilities due to context windows, but I wouldn't be surprised if it could do short stories that pass this Turing test right now.
Post reply on HN