Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

arxiv.org

Post reply on HN
Print view View original

231–240 of 387 posts

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#231

Post by ajam1507 »

9dev wrote: post #124
concinds wrote: post #122

Earlier quoted context omitted.

> Claude refused to build me a news scraper that would post political hot takes to twitter > Just choose your political leaning and the AI would consult the correct echo chambers to repeat from. You're effectively asking it to build a social media political manipulation bot, behaviorally identical to the bots that propagandists would create. Shows that those guardrails can be ineffective and trivial to bypass.

> Good illustration that those guardrails are ineffective and trivial to bypass. Is that genuinely surprising to anyone? The same applies to humans, really—if they don't see the full picture, and their individual contribution seems harmless, they will mostly do as told. Asking critical questions is a rare trait. I would argue its completely futile to even work on guardrails, if defeating them is just a matter of refr…

> I would argue its completely futile to even work on guardrails

Maybe if humans were the only ones prompting AI models

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#232

Post by WillAdams »

RupertSalt wrote: post #216
WillAdams wrote: post #215

Earlier quoted context omitted.

From an IBM training manual (1979): >A computer can never be held accountable >Therefore a computer must never make a management decision The (EDITED) corollary would arguably be: >Corporations are amoral entities which are potentially immortal who cannot be placed behind bars. Therefore they should never be given the rights of human beings. (potentially, not absolutely immortal --- would wording as "not mortal by es…

How is a corporation "immortal"? What is the oldest corporation in the world? I mean, aside from churches and stuff. Corporations can die or be killed in numerous ways. Not many of them will live forever. Most will barely outlive a normal human's lifespan. By definition, since a corporation comprises a group of people, it could never outlive the members, should they all die at some point. Let us also draw a distincti…

Construction company okay?

>Kongo Gumi, founded in 578 AD, is recognized as the oldest continuously operating company in the world, specializing in the construction of Buddhist temples.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#233

Post by psychoslave »

From my experience, if LLMs prose output was generated by some human, they would easily fall in the worst sociopath class one can interact with. Filling all the space with 99% blatant lies in the most confident way. In comparison, even top percentile of human hierarchies feels like a class of shy people fully dictated to staying true and honest in all situations.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#234

Post by IanCal »

cyanydeez wrote: post #212
IanCal wrote: post #183

Earlier quoted context omitted.

A remarkable number of humans given really quite basic feedback will perform actions they know will very directly hurt or kill people. There are a lot of critiques about quite how to interpret the results but in this context it’s pretty clear lots of humans can be at least coerced into doing something extremely unethical. Start removing the harm one, two, three degrees and add personal incentives and is it that surpr…

> 2012, Australian psychologist Gina Perry investigated Milgram's data and writings and concluded that Milgram had manipulated the results, and that there was a "troubling mismatch between (published) descriptions of the experiment and evidence of what actually transpired." She wrote that "only half of the people who undertook the experiment fully believed it was real and of those, 66% disobeyed the experimenter".[29…

What you have quoted says a third of people who thought it was real didn’t disobey the experimenter when they thought they were delivering dangerous and lethal electric shocks to a human. Is that correct?

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#235

Post by lnenad »

coldtea wrote: post #223
bonesss wrote: post #207

Earlier quoted context omitted.

I addressed that directly in the comment you’re replying to. It’s understandable people readily anthropomorphize algorithmic output designed to provoke anthropomorphized responses. It is not desire-able, safe, logical, or rational since (to paraphrase:), they are complex text transformation algorithms that can, at best, emulate training data reinforced by benchmarks and they display emergent behaviours based on those…

> It is not desire-able, safe, logical, or rational since (to paraphrase:), they are complex text transformation algorithms that can, at best, emulate training data reinforced by benchmarks and they display emergent behaviours based on those. > They are not human, so attributing human characteristics to them is highly illogical Nothing illogical about it. We attribute human characterists when we see human-like behavi…

I agree 100% with everything you wrote.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#236

Post by chrononaut »

IanCal wrote: post #183
badgersnake wrote: post #158

Earlier quoted context omitted.

Humans risk jail time, AIs not so much.

A remarkable number of humans given really quite basic feedback will perform actions they know will very directly hurt or kill people. There are a lot of critiques about quite how to interpret the results but in this context it’s pretty clear lots of humans can be at least coerced into doing something extremely unethical. Start removing the harm one, two, three degrees and add personal incentives and is it that surpr…

Normalization of deviance also contributes towards unethical outcomes, where people would not have selected that outcome originally.

https://en.wikipedia.org/wiki/Normalization_of_deviance

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#238

Post by [deleted] »

RupertSalt wrote: post #228
skeptic_ai wrote: post #222

Earlier quoted context omitted.

What needs to do a company from fortune 7 to die? If kills 1 person they won’t close Google. If steals 1 billion, won’t close either. So what needs to do such a company to be closed down? I think it’s almost impossible to shut down

Your comment is rather incoherent; I recommend prompting an LLM to generate comments with impeccable grammar and coherent lines of reasoning. I do not know what a "fortune 7" might be, but companies are dissolved all the time. Thousands per year, just administratively. For example, notable incidents from the 21st c: Arthur Andersen, The Trump Foundation, Enron, and Theranos are all entities which were completely liqu…

[deleted]

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#239

Post by data-ottawa »

NiloCK wrote: post #22
hypron wrote: post #5

https://i.imgur.com/23YeIDo.png Claude at 1.3% and Gemini at 71.4% is quite the range

This comment is too general and probably unfair, but my experience so far is that Gemini 3 is slightly unhinged. Excellent reasoning and synthesis of large contexts, pretty strong code, just awful decisions. It's like a frontier model trained only on r/atbge. Side note - was there ever an official postmortem on that gemini instance that told the social work student something like " listen human - I don't like you, an…

Gemini 3 (Flash & Pro) seemingly will _always_ try and answer your question with what you give it, which I’m assuming is what drives the mentioned ethics violations/“unhinged” behaviour.

Gemini’s strength definitely is that it can use that whole large context window, and it’s the first Gemini model to write acceptable SQL. But I agree completely at being awful at decisions.

I’ve been building a data-agent tool (similar to [1][2]). Gemini 3’s main failure cases are that it makes up metrics that really are not appropriate, and it will use inappropriate data and force it into a conclusion. When a task is clear + possible then it’s amazing. When a task is hard with multiple failure paths then you run into Gemini powering through to get an answer.

Temperature seems to play a huge role in Gemini’s decision quality from what I see in my evals, so you can probably tune it to get better answers but I don’t have the recipe yet.

Claude 4+ (Opus & Sonnet) family have been much more honest, but the short context windows really hurt on these analytical use cases, plus it can over-focus on minutia and needs to be course corrected. ChatGPT looks okay but I have not tested it. I’ve been pretty frustrated at ChatGPT models acting one way in the dev console and completely different in production.

[1] https://openai.com/index/inside-our-in-house-data-agent/ [2] https://docs.cloud.google.com/bigquery/docs/conversational-a...

Post reply on HN