Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The other half of AI safety

personalaisafety.com

Post reply on HN
Print view View original

31–40 of 137 posts

Re: The other half of AI safety

#31

Post by zdragnar »

api wrote: post #15
Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

If anything, my use of AI (admittedly not as a companion or a psychologist) suggests that it is on the whole significantly less toxic than the seething cess pit of social media. AI is positively affirming by comparison.

That's why it is dangerous to some- it is an enabler, and will feed things that should not be fed.

Social media is like this too. They can both be bad.

Re: The other half of AI safety

#32

Post by concinds »

nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

"Routed to a human" is what the suicide hotline numbers do. OpenAI employees are neither trained nor credible to do that stuff.

Re: The other half of AI safety

#33

Post by asdff »

ngruhn wrote: post #4

The bad cases make headlines. But I think it's quite possible that AI is helping a lot of people in distress. Many people are uncomfortable opening up to humans, or have no one to talk to, or can't afford to fork over whatever-hourly-rate a therapist takes.

Therapy is cheap (as in like $10)/free with insurance. However there are still 10 states that have not expanded medicaid after the ACA, mostly in the south.

But also, to suggest these people are not receiving therapy is not always the case. Talk therapy is just that, talking to someone on ones problems to learn about them, their triggers, determining coping mechanisms to move forward with one's life. People might instead be getting all that from their barber, drinking buddy, or their priest, rather than in a 1 hour appointment with a therapist.

Re: The other half of AI safety

#34

Post by Gigachad »

nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

Tech companies will pull trillions of dollars out of their asses when the problem is boosting ad revenue or automating people out of a job. But when asked to deal with the crisis they invented and dumped on society the answer is “that’s impossible, doesn’t scale”

Re: The other half of AI safety

#35

Post by chromacity »

SilverElfin wrote: post #22
autoexec wrote: post #16

Earlier quoted context omitted.

> I don't buy that chatGPT is actually doing these users any harm. I have zero doubt that chatgpt is doing users harm. I even give chatgpt a pass on giving vulnerable people, including children, instructions and information about how to kill themselves. One place chatgpt goes over the line is actively encouraging them to go through with suicide. I also don't doubt that it feeds into mania and psychosis. While almost…

If it wasn’t ChatGPT but a fiction book, would you feel the author is “doing harm”? Or is the reader doing it to themselves?

If that book was titled "hey mentally ill person, you should kill yourself", and if I was handing it out in front of a clinic, then yes, I'd probably bear some blame.

Normal, well-adjusted people have genuine difficulty understanding the boundaries of this tech specifically because it's designed to be sycophantic and human-like. They ask AI for life and career advice, use it for therapy, ask it to interpret dreams, develop romantic relationships with AI "girlfriends", etc. I had two friends who believed they are "exploring the frontiers of science" with ChatGPT while spiraling into the depths of quantum multidimensional gobbledygook.

I'll give you that some on this is on us because we just don't know how to deal with a "human-shaped" conversation partner that isn't human and has no trouble praising Hitler if you prompt it the right way. But if you're building a billion- or trillion-dollar empire on top of it, you don't get to wash your hands clean.

Re: The other half of AI safety

#36

Post by photochemsyn »

The ‘tobacco warning label’ approach sounds good but I’m not sure if it stopped that many people from smoking or was just a means for corporations to limit their liability. Corporate culture being what it is, having warnings like the following pop up every time a client opens an LLM app would not be that popular in the C-suite. Possible examples:

AI MENTAL SAFETY WARNING:

> This chatbot can sound caring, certain, and personal, but it is not a human and cannot protect your mental health. It may reinforce false beliefs, emotional dependence, suicidal thinking, manic plans, paranoia, or poor decisions. Do not use it as your therapist, only confidant, crisis counselor, doctor, lawyer, or source of reality-testing.

AI TECHNICAL SAFETY WARNING

> This AI may generate plausible but destructive technical instructions. Incorrect commands can erase data, expose secrets, compromise security, damage systems, or brick hardware. Never run commands you do not understand. Always verify AI-generated code, scripts, and shell commands before execution.

Now, if I’m running my own open-source model on my own hardware, I can’t really blame the model if I myself make bad decisions based on its advice - that’s like growing your own tobacco from seed in your garden, drying and curing it, then complaining about the health effects after you smoke it. If I give it agentic capabilities on my LAN without understanding the risks, same old story - with great power comes great responsibility.

Re: The other half of AI safety

#38

Post by b00ty4breakfast »

BobbyJo wrote: post #17
Turskarama wrote: post #11

Earlier quoted context omitted.

Just because the users were already sick when they started using ChatGPT doesn't mean that ChatGPT isn't exacerbating the issue. Sickness isn't a boolean condition. A big problem with LLMs in general when it comes to people like this is that they are too sycophantic, they don't push back when you start acting strange and they're too gentle about trying to validate you.

It's hyper palatable food in the form of conversation. I see society treating it the same way eventually, at least along this one axis of interaction.

boy, if we treat it like junk food, things are only going to get worse for some places in the world. The food over here in the states is pretty awful if you aren't paying attention. Sugar in everything, high calorie/low nutrition etc.

Re: The other half of AI safety

#39

Post by hx8 »

nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

I don't think it's obvious that routing to a human is infeasible. I'm sure many local authorities, health agencies, and non-profits would be okay being routed to. Additionally, I'm sure many of the users are the same week over week, so giving them long term care would reduce the total volume. Finally, there is a long gap between psychosis and emotional dependence, so there could be some triage to make sure those most in need have human intervention.

Re: The other half of AI safety

#40

Post by busterarm »

b00ty4breakfast wrote: post #37
Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

[flagged]

Unfortunately, mental disabilities are a protected class. You can't do a mental health evaluation without giving it to everyone in the company and even then you can't do anything discriminatory with the results.

You have to prove that the person is going to cause immediate direct harm to their coworkers before you can really do anything and that's difficult and expensive to prove.

Post reply on HN