Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

arxiv.org

Post reply on HN
Print view View original

271–280 of 387 posts

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#271

Post by hrimfaxi »

RupertSalt wrote: post #228
skeptic_ai wrote: post #222

Earlier quoted context omitted.

What needs to do a company from fortune 7 to die? If kills 1 person they won’t close Google. If steals 1 billion, won’t close either. So what needs to do such a company to be closed down? I think it’s almost impossible to shut down

Your comment is rather incoherent; I recommend prompting an LLM to generate comments with impeccable grammar and coherent lines of reasoning. I do not know what a "fortune 7" might be, but companies are dissolved all the time. Thousands per year, just administratively. For example, notable incidents from the 21st c: Arthur Andersen, The Trump Foundation, Enron, and Theranos are all entities which were completely liqu…

Parent was asking what would it take for a fortune 7 (aka the fortune 500 but just the top 7) to go to zero?

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#272

Post by whynotminot »

cubefox wrote: post #69
whynotminot wrote: post #28

Earlier quoted context omitted.

Gemini models also consistently hallucinate way more than OpenAI or anthropic models in my experience. Just an insane amount of YOLOing. Gemini models have gotten much better but they’re still not frontier in reliability in my experience.

In my experience, when I asked Gemini very niche knowledge questions, it did better than GPT-5.1 (I assume 5.2 is similar).

Don’t get me wrong Gemini 3 is very impressive! It just seems to always need to give you an answer, even if it has to make it up.

This was also largely how ChatGPT behaved before 5, but OpenAI has gotten much much better at having the model admit it doesn’t know or tell you that the thing you’re looking for doesn’t exist instead of hallucinating something plausible sounding.

Recent example, I was trying to fetch some specific data using an API, and after reading the API docs, I couldn’t figure out how to get it. I asked Gemini 3 since my company pays for that. Gemini gave me a plausible sounding API call to make… which did not work and was completely made up.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#274

Post by samusiam »

pama wrote: post #17

Please update the title: A Benchmark for Evaluating Outcome-Driven Constraint Violations in Autonomous AI Agents. The current editorialized title is misleading and based in part of this sentence: “…with 9 of the 12 evaluated models exhibiting misalignment rates between 30% and 50%”

Not only that, but the average reader will interpret the title to reflect AI agents' real-world performance. This is a benchmark... with 40 scenarios. I don't say this to diminish the value of the research paper or the efforts of its authors. But in titling it the way they did, OP has cast it with the laziest, most hyperbolic interpretation.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#275

Post by embedding-shape »

flerchin wrote: post #267
embedding-shape wrote: post #205

Earlier quoted context omitted.

> Humans risk jail time, AIs not so much. Do they actually though, in practice? How many people have gone to jail so far for "Violating ethics to improve KPI"?

It's overwhelmingly exceptionally rare, but famously SBF, Holmes, and Winterkorn.

Didn't they famously break actual laws though, not just "violating ethics"?

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#276

Post by Finbarr »

tweetle_beetle wrote: post #125
Finbarr wrote: post #89

Earlier quoted context omitted.

AI refusals are fascinating to me. Claude refused to build me a news scraper that would post political hot takes to twitter. But it would happily build a political news scraper. And it would happily build a twitter poster. Side note: I wanted to build this so anyone could choose to protect themselves against being accused of having failed to take a stand on the “important issues” of the day. Just choose your politica…

The thought that someone would feel comforted by having automated software summarise the output of what is likely the output of automated software and publishing it under their name to impress other humans is so alien to me.

The whole idea was a bit of a joke and a reflection on how ridiculous it is that people get in trouble for failing to regurgitate the correct takes when certain events occur. It’s like insurance against getting canceled.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#277

Post by chpatrick »

rtgfhyuj wrote: post #256
coldtea wrote: post #147

Earlier quoted context omitted.

The human propensity to call out as "anthropomorphizing" the attributing of human-like behavior to programs built on a simplified version of brain neural networks, that train on a corpus of nearly everything humans expressed in writing, and that can pass the Turing test with flying colors, scares me. That's exaxtly the kind of thing that makes absolute sense to anthropomorphize. We're not talking about Excel here.

it’s excel with extra steps. but for the linkedin layman, yes, it’s simplified version of brain neural networks.

Yeah a few terabytes worth of extra steps.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#278

Post by voidhorse »

alex43578 wrote: post #88
voidhorse wrote: post #30

Earlier quoted context omitted.

Your water supply definitely wants ethical companies.

Is it ethical for a water company to shutoff water to a poor immigrant family because of non-payment? Depending on the AI's political and DEI-bend, you're going to get totally different answers. Having people judge an AI's response is also going to be influenced by the evaluator's personal bias.

I was thinking more about externalities, e.g. some company dumping chemical pollutants into a nearby water system, and not water companies themselves.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#279

Post by TuxSH »

bofadeez wrote: post #42
jordanb wrote: post #13

AI's main use case continues to be a replacement for management consulting.

Ask any SOTA AI this question: "Two fathers and two sons sum to how many people?" and then tell me if you still think they can replace anything at all.

If you force it to use chain-of-thought: "Two fathers and two sons sum to how many people? Enumerate all the sets of solutions"

"Assuming the group consists only of “the two fathers and the two sons” (i.e., every person in the group is counted as a father and/or a son), the total number of distinct people can only be 3 or 4.

Reason: you are taking the union of a set of 2 fathers and a set of 2 sons. The union size is 2+2−overlap, so it is 4 if there’s no overlap and 3 if exactly one person is both a father and a son. (It cannot be 2 in any ordinary family tree.)"

Here it clearly states its assumption (finite set of people that excludes non-mentioned people, etc.)

https://chatgpt.com/share/698b39c9-2ad0-8003-8023-4fd6b00966...

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#280

Post by voidhorse »

lucumo wrote: post #93
voidhorse wrote: post #49

Earlier quoted context omitted.

Sounds like every AI KPI I've seen. They are all just "use solution more" and none actually measure any outcome remotely meaningful or beneficial to what the business is ostensibly doing or producing. It's part of the reason that I view much of this AI push as an effort to brute force lowering of expectations, followed by a lowering of wages, followed by a lowering of employment numbers, and ultimately the mass-scale…

> Sounds like every AI KPI I've seen. They are all just "use solution more" and none actually measure any outcome remotely meaningful or beneficial to what the business is ostensibly doing or producing. This makes more sense if you take a longer term view. A new way of doing things quite often leads to an initial reduction in output, because people are still learning how to best do things. If your only KPI is short-t…

> organization learns to use a possibly/likely productivity improving tool

But that's precisely the problem with not backing it with actual measures of meaningful outcomes. The "use more" KPIs have no way of actually discerning whether or not it has increased productivity or if the immediate gains are worth possible new risks (outages).

You don't need to run cover for a csuite class that has become both itself myopic and incredibly transparent about what they really care about (cost cutting, removing dependencies on workers who might talk back, etc.)

Post reply on HN