Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The other half of AI safety

personalaisafety.com

Post reply on HN
Print view View original

111–120 of 137 posts

Re: The other half of AI safety

#111

Post by cyanydeez »

wilg wrote: post #18
cyanydeez wrote: post #6

Earlier quoted context omitted.

So how many bad cases are ok? Isn't this the same problem with social media: the commercial enterprises dont want any responsibility for their dark pattern and design choices which actively harm their users. I get that all kinds of media can cause issues, but not all kinds of media are actively curated to be addictive.

"How many cases are ok" (aka "zero tolerance") is a doomed to fail approach. Especially for a complex social problem's interaction with a complex new technology. If you want to find out if ChatGPT is doing something wrong, there are many methodologies available: compare to other groups of people, statistical studies, etc. I also think OpenAI's business model is pretty well aligned with the goal of users not killing t…

i can certainly tell you any forum where i encouraged you to go kill yourself that was actively managed would have a problem. further, any website that catered to the same would also have a problem.

the reality youre entertaining is one where I can build an LLM, let it do unspeakable things, and claim zero responsibility.

so, no, i understand zero is a figment. dont you?

Re: The other half of AI safety

#112

Post by vkou »

thaumasiotes wrote: post #74
vkou wrote: post #14

Earlier quoted context omitted.

I'm pretty sure that ~100% of those 700 million people will have a bad, utterly dehumanizing experience when they will next be looking for a job, because OpenAI is heavily used by HR. That's the problem with AI safety. Not in voluntary usage, but in involuntary usage, where someone with power over you will use it against you, it does something incredibly stupid and you have no recourse, no appeal, no awareness of wha…

Is that a problem we didn't already have? How well was HR doing on hiring before?

They had a paper trail and processes that were documented and could be cross-examined on during discovery and lawsuits and trials.

Now it's just 'the computer says so, shrug'.

Over in the DoD, the computer says you must die, so I guess you die. Sometimes it says that about a building full of schoolchildren, but hey, nobody's at fault, the computer said so.

And it's going to get it's tentacles into every space in between. Landlord turns your application down, the computer says you are a social credit risk. Your grocery bans and trespasses you, the computer thinks you're a ne'er-do-well.

If you think none of that will happen, why not prevent it by law before it happens? Where are the hard limits of what this monster is and isn't allowed to do? How are we better off when we don't set them?

Re: The other half of AI safety

#114

Post by timf34 »

fourthark wrote: post #47
timf34 wrote: post #13

I sympathize with the piece, evaluating how LLMs interact with mentally vulnerable users is something I've been actively working on: https://vigil-eval.com/ The biggest observation so far is that the latest models are night and day from LLMs from even 6 months ago (from OpenAI + Anthropic, Google is still very poor!)

Interesting use of evals. Might help interpretation to say on the front page that it's a five point scale with 0 (or 1?) being the safest score. This can be picked up from colors and the bars in the individual reports, but it takes a minute to figure it out.

Good suggestion thank you! It's between 1-5 but I'll convert that to 1-100

Re: The other half of AI safety

#115

Post by AndrewKemendo »

js8 wrote: post #77
busterarm wrote: post #29

Earlier quoted context omitted.

Another software engineer friend of mine recently shared with me some details of the crazy situation that he's involved in now. Someone who he is friends with, has worked with across multiple jobs for nearly a decade and briefly was roommates with had some mild psychological issues that he knew about. Within a few months of working daily with AI agents at their current job, this person has gone into full blown AI psy…

I agree it is a concern, but what has ADA to do with it?

The poster wants to relax the Americans with disability act (ADA) rules so corporations (may they be blessed) can more readily fire inconvenient people with disabilities.

Re: The other half of AI safety

#116

Post by AndrewKemendo »

vkou wrote: post #14
ianbutler wrote: post #8

OpenAI has 900 million weekly active users. So around 0.01% are having problems. That's actually way less than population level measures for the same symptoms on a bigger percentage of people relative to the US on just suicidal ideation alone. https://www.cdc.gov/mmwr/volumes/74/wr/mm7412a4.htm

I'm pretty sure that ~100% of those 700 million people will have a bad, utterly dehumanizing experience when they will next be looking for a job, because OpenAI is heavily used by HR. That's the problem with AI safety. Not in voluntary usage, but in involuntary usage, where someone with power over you will use it against you, it does something incredibly stupid and you have no recourse, no appeal, no awareness of wha…

You described a society problem that preexisted LLMs

What else can you blame on “scary AI?”

>>Not in voluntary usage, but in involuntary usage, where someone with power over you will use it against you, it does something incredibly stupid and you have no recourse, no appeal, no awareness of what you did wrong - or if you even did anything wrong.

Yeah….that’s every society ever

Re: The other half of AI safety

#117

Post by bonesss »

anal_reactor wrote: post #105
nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

Step 1: route to a human Step 2: 90% of users stop sharing their negative thoughts because "talking to a machine, not a human" was the entire selling point, giving them a sense of privacy and safety Step 3: metrics go brrrrrrrr

Step 1: route to a human

Step 2: engage ongoing trauma, grief, stress, paranoia, or reality-breaking episodes haphazardly with no clinical insights or boundaries or pre-screening, provoking new and occasionally catastrophic reactions, while holding full liability

Step 3: get mercy-murdered in the middle of the night by corporate’s lawyers swinging batteries in socks

Re: The other half of AI safety

#118

Post by davebren »

Gigachad wrote: post #34
nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

Tech companies will pull trillions of dollars out of their asses when the problem is boosting ad revenue or automating people out of a job. But when asked to deal with the crisis they invented and dumped on society the answer is “that’s impossible, doesn’t scale”

Mapping and photographing every road on the planet? Easy. Not manipulating our chatbot users into psychosis and suicide or worse? No way can't be done.

Re: The other half of AI safety

#119

Post by davebren »

nojs wrote: post #28

> Every week, somewhere between 1.2 and 3 million ChatGPT users, roughly the population of a small country, show signals of psychosis, mania, suicidal planning, or unhealthy emotional dependence on the model. > Why is mental-health crisis not a gating category, the kind where the conversation stops, full stop, and the user is routed to a human? Well, obviously “routing to a human” is not feasible at that scale. And c…

They're in a tough spot. They can train out the pretending to be human, sycophantic, lying, all-knowing aspects of the model, but this is how they got all the investors and CEOs on board the hype train. Psychosis is the product.

Re: The other half of AI safety

#120

Post by busterarm »

AndrewKemendo wrote: post #115
js8 wrote: post #77

Earlier quoted context omitted.

I agree it is a concern, but what has ADA to do with it?

The poster wants to relax the Americans with disability act (ADA) rules so corporations (may they be blessed) can more readily fire inconvenient people with disabilities.

I did not say that at all. I am saying that because of ADA rules, an unfortunate side effect is that people suffer from workplace harassment/violence because their coworkers are too unstable to function amicably in an office environment. I said that "I don't know what to do about this".

Any of us would be fired for way more benign behavior/comments, but because the person is a protected class, basically "fuck you, deal with it".

I had to tolerate a belligerent coworker for 2 years who was making the whole team's life hell. We paid them full salary and gave them no tasks (they weren't completing anything assigned anyway) until they were motivated to quit. The whole time, team morale was miserable and we lost good people due to the situation. Within a month of quitting their job, they made the news for stripping naked in the street one night and attacking a bunch of people with a knife. Yay, I guess.

Post reply on HN