Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The other half of AI safety

personalaisafety.com

Post reply on HN
Print view View original

11–20 of 137 posts

Re: The other half of AI safety

#11

Post by Turskarama »

Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

Just because the users were already sick when they started using ChatGPT doesn't mean that ChatGPT isn't exacerbating the issue. Sickness isn't a boolean condition. A big problem with LLMs in general when it comes to people like this is that they are too sycophantic, they don't push back when you start acting strange and they're too gentle about trying to validate you.

Re: The other half of AI safety

#13

Post by timf34 »

I sympathize with the piece, evaluating how LLMs interact with mentally vulnerable users is something I've been actively working on: https://vigil-eval.com/

The biggest observation so far is that the latest models are night and day from LLMs from even 6 months ago (from OpenAI + Anthropic, Google is still very poor!)

Re: The other half of AI safety

#14

Post by vkou »

ianbutler wrote: post #8

OpenAI has 900 million weekly active users. So around 0.01% are having problems. That's actually way less than population level measures for the same symptoms on a bigger percentage of people relative to the US on just suicidal ideation alone. https://www.cdc.gov/mmwr/volumes/74/wr/mm7412a4.htm

I'm pretty sure that ~100% of those 700 million people will have a bad, utterly dehumanizing experience when they will next be looking for a job, because OpenAI is heavily used by HR.

That's the problem with AI safety. Not in voluntary usage, but in involuntary usage, where someone with power over you will use it against you, it does something incredibly stupid and you have no recourse, no appeal, no awareness of what you did wrong - or if you even did anything wrong.

And it's not just employment. Governments, vendors, retailers, landlords, utilities are, or will all be using it in situations that will dramatically impact your life.

Re: The other half of AI safety

#15

Post by api »

Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

If anything, my use of AI (admittedly not as a companion or a psychologist) suggests that it is on the whole significantly less toxic than the seething cess pit of social media.

AI is positively affirming by comparison.

Re: The other half of AI safety

#16

Post by autoexec »

Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

> I don't buy that chatGPT is actually doing these users any harm.

I have zero doubt that chatgpt is doing users harm. I even give chatgpt a pass on giving vulnerable people, including children, instructions and information about how to kill themselves. One place chatgpt goes over the line is actively encouraging them to go through with suicide.

I also don't doubt that it feeds into mania and psychosis. While almost anything can do the same, they've designed the service to be as addictive and engaging as possible in part by turning up the ass-kissing sycophancy to 11 with total disregard for the fact that there are times when it's very dangerous to encourage and support everything someone says no matter how obviously sick they are. They also want to whore themselves out as a virtual therapist while being unfit and unqualified for the job and that's just one of many roles the chatbot isn't fit for but they're happy to let you try anyway.

Re: The other half of AI safety

#17

Post by BobbyJo »

Turskarama wrote: post #11
Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

Just because the users were already sick when they started using ChatGPT doesn't mean that ChatGPT isn't exacerbating the issue. Sickness isn't a boolean condition. A big problem with LLMs in general when it comes to people like this is that they are too sycophantic, they don't push back when you start acting strange and they're too gentle about trying to validate you.

It's hyper palatable food in the form of conversation. I see society treating it the same way eventually, at least along this one axis of interaction.

Re: The other half of AI safety

#18

Post by wilg »

cyanydeez wrote: post #6
ngruhn wrote: post #4

The bad cases make headlines. But I think it's quite possible that AI is helping a lot of people in distress. Many people are uncomfortable opening up to humans, or have no one to talk to, or can't afford to fork over whatever-hourly-rate a therapist takes.

So how many bad cases are ok? Isn't this the same problem with social media: the commercial enterprises dont want any responsibility for their dark pattern and design choices which actively harm their users. I get that all kinds of media can cause issues, but not all kinds of media are actively curated to be addictive.

"How many cases are ok" (aka "zero tolerance") is a doomed to fail approach. Especially for a complex social problem's interaction with a complex new technology.

If you want to find out if ChatGPT is doing something wrong, there are many methodologies available: compare to other groups of people, statistical studies, etc.

I also think OpenAI's business model is pretty well aligned with the goal of users not killing themselves for like 100 reasons. And they do appear to take it seriously.

Re: The other half of AI safety

#19

Post by davorak »

Legend2440 wrote: post #7

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

> I don't buy that chatGPT is actually doing these users any harm.

For me to buy this as true I would expect that those people would be as well off or as bad off if chatGPT was in their life or not.

I expect that some people are worse off with chatGPT in their life.

Responsibility for that harm is a different question though. Some people are also better of without cars in their life and we let the government laws sort that out.

Getting openAI and similar companies to act in mitigating these harms serves at least a few purposes; reducing the overall harm in the world, reducing/limiting future government regulation, maximizing the adoption of ai tools, potentially increasing long term profits of the companies in question.

Re: The other half of AI safety

#20

Post by falcor84 »

simonw wrote: post #2

"There is no independent audit, no time series, no disclosed methodology, so we have no idea whether the real figure is higher, whether it is growing, or how it compares across the other frontier models, none of which publish equivalent data." Tip for writers: aggressively filter out the "no X, no Y, no Z" pattern from your writing. Whether or not you used AI to help you write it's such a red flag now that you should…

Why is it a red flag?

How is it different from any other purely stylistic rules such as Strunk and White's prohibitions against split infinitives and the passive voice, which we've left far behind us? Why shouldn't people just write however feels natural to them as long as the message is clear?

Post reply on HN