Live data from Hacker News

OpenAI says over a million people talk to ChatGPT about suicide weekly

techcrunch.com

271–280 of 585 posts

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#271
The bar for medical devices in most countries is _incredibly_ high, for good reason. ChatGPT wasn't developed with the idea of being a therapist in mind, it was a side-effect of the technology that was developed.

Why is OpenAI getting a free pass here?

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#272

This is not suprising at all. Having gone through therapy a few years back, I would have had a chat with LLMs if I was in a poor mental health situation. There is no other system that is available at scale, 24x7 on my phone. A chat like this is not a solution though, it is an indicator that our societies have issues is large parts of our population that we are unable to deal with. We are not helping enough people. To…

I choose to believe that too. I think more people are interested than we’d initially believe. Money restrains many of our true wants. Sidebar — I do sympathize with the problem being thrust upon them, but it is now theirs to either solve or refuse. A chat like this is all you’ve said and dangerous, because they play a middle ground: Presenting a machine can evaluate your personal situation and reason about it, when i…

Sorry - what's with "I choose to believe"? Either you believe something or not, there is no choice. Maybe you mean "hope". Or "wishful thinking".

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#273
post #157

Contrarian opinion. OpenAI gets a lot of hate these days, but on this subject it's quite possible that ChatGPT helped a lot of people choose a less drastic path. There could have been unfortunate incidents, but the number of people who were convinced to not take extreme steps would have been of a few orders of magnitude more (guessing). I use it to help improve mental health, and with good prompting skills it's not b…

"ChatGPT saved me from years of suicidal thoughts in DAYS": https://old.reddit.com/r/traumatoolbox/comments/1kdx3aw/chat...

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#274
post #254

Earlier quoted context omitted.

I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No. By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much . Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense". Now this part: > Add in the commercial incentives of 'O…

While I agree with most of your comment, I'd like to dispute the story about GPT-2. Yes, they did claim that they wouldn't release GPT-2 due to unforeseen risks, but... a. they did end up releasing it, b. they explicitly stated that they wouldn't release GPT-3[1] for marketing/financial reasons, and c. it being dangerous didn't stop them from offering the service for a profit. I think the quotes around "open" are wel…

> they did end up releasing it,

After studying it extensively with real-world feedback. From everything I've seen, the statement wasn't "will never release", it was vaguer than that.

> they explicitly stated that they wouldn't release GPT-3 for marketing/financial reasons

Not seen this, can you give a link?

> it being dangerous didn't stop them from offering the service for a profit.

Please do be cynical about how honest they were being — I mean, look at the whole of Big Tech right now — but the story they gave was self-consistent:

[Paraphrased!] (a) "We do research" (they do), "This research costs a lot of money" (it does), and (b) "As software devs, we all know what 'agile' is and how that keeps product aligned with stakeholder interest." (they do) "And the world is our stakeholder, so we need to release updates for the world to give us feedback." (???)

That last bit may be wishful thinking, I don't want to give the false impression that I think they can do no wrong (I've been let down by such optimism a few other times), but it is my impression of what they were claiming.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#275

Earlier quoted context omitted.

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

> Supposing that the advice it provides does more good than harm

That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument.

I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the problem? The supposition is wrong and the suggestion is dangerous, but by framing it as “supposing” with a made up positive outcome, we make it sound non-threatening and reasonable.

Or for a more realistic example, “suppose drinking bleach could cure COVID-19”.

First understand if the idea has the potential to do the thing, only then (with considerably more context) consider if it’s worth implementing.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#276
post #59
post #8

Earlier quoted context omitted.

Are you sure ChatGPT is the solution? It just sounds like another "savior complex" sell spin from tech. 1. Social media -> connection 2. AGI -> erotica 3. Suicide -> prevention All these for engagement (i.e. addiction). It seems like the tech industry is the root cause itself trying to masquerade the problem by brainwashing the population. https://news.ycombinator.com/item?id=45026886

I agree that the tech industry is the root cause of a lot of mental illness. But social media is a far bigger concern than AI. Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion...

[deleted]

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#277
I think the major issue with asking LLMs (CGPT, etc.) for advice on various subjects is that they are typically 80-90% accurate. YMMV, speaking anecdotally here. Which means that the chance of them being wrong becomes an afterthought. You know there's a chance of that, but not bothering to verify the answer leads to an efficiency that rarely bites you. And if you stop verifying the answers, incorrect ones may go unnoticed, further obscuring the risk of that practice.

It's a hard thing to solve. I wouldn't expect LLM providers to care because that's how our (current) society works, and I wouldn't expect users to know better because that's how most humans operate.

If anyone has a good idea for this, I'm open to suggestions.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#278
post #275

Earlier quoted context omitted.

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

> Supposing that the advice it provides does more good than harm That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument. I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the proble…

In my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.

Re: OpenAI says over a million people talk to ChatGPT about suicide weekly

#279
post #198
post #191

Earlier quoted context omitted.

I didn't think marriage was that bad but point taken!

I think GP meant that "commit" is often used in the phrase "commit suicide". That wasn't a comment about relationship commitment.

Yeah, they were joking.
Post reply on HN