Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

arxiv.org

Post reply on HN
Print view View original

321–330 of 387 posts

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#321

Post by stingraycharles »

RobotToaster wrote: post #151
alentred wrote: post #133

If we abstract out the notion of "ethical constraints" and "KPIs" and look at the issue from a low-level LLM point of view, I think it is very likely that what these tests verified is a combination of: 1) the ability of the models to follow the prompt with conflicting constraints, and 2) their built-in weights in case of the SAMR metric as defined in the paper. Essentially the models are given a set of conflicting co…

It would also be interesting to see how humans perform on the same kind of tests. Violating ethics to improve KPI sounds like your average fortune 500 business.

That really doesn’t matter a lot. The reason why it’s important for AIs to follow these rules is that it’s important for them to operate within a constrained set of rules. You can’t guarantee that programmatically, so you try to prove that it can be done empirically as a proxy.

AIs can be used and abused in ways that are entirely different from humans, and that creates a liability.

I think it’s going to be very difficult to categorically prevent these types of issues, unless someone is able to integrate some truly binary logic into LLM systems. Which is nearly impossible, almost by definition of what LLMs are.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#322

Post by yread »

Washuu wrote: post #145
neya wrote: post #130

Earlier quoted context omitted.

I completely disagree. Gemini is by far the most straightforward AI. The other two are too soft. ChatGPT particularly is extremely politically correct all the time. It won't call a spade, one. Gemini has even insulted me - just to get my ass moving on a task when givn the freedom. Which is exactly what you need at times. Not constant ass kissing "ooh your majesty" like ChatGPT does. Claude has a very good balance whe…

Using Gemini 3 Pro Preview, it told me in mostly polite terms, that I'm a fucking idiot. Like I would expect a close friend to do when I'm going about something wrong. ChatGPT with the same prompt tried to do whatever it would take to please me to make my incorrect process work.

I got the same but it was wrong

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#323

Post by pixl97 »

mrguyorama wrote: post #320
coldtea wrote: post #150

Earlier quoted context omitted.

Obviously, why? Because it makes calculations? You think that ultimately your brain doesn't also make calculations as its fundamental mechanism? The architecture and substrate might be different, but they are calculations all the same.

Brains do not "make calculations". Biological neurons do not "make calculations" What they do is well described by a bunch of math. You've got the direction of the arrow backwards. Map, territory, etc.

So what does a chemical based computer do?

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#325

Post by ajcp »

nradov wrote: post #297
rogerkirkness wrote: post #293

We're a startup working on aligning goals and decisions and agentic AI. We stopped experimenting with decision support agents, because when you get into multiple layers of agents and subagents, the subagents would do incredibly unethical, illegal or misguided things in service of the goal of the original agent. It would use the full force of reasoning ability it had to obscure this from the user. In a sense, it was n…

Illegal? Seriously? What specific crimes did they commit? Frankly I don't believe you. I think you're exaggerating. Let's see the logs. Put up or shut up.

Fraud is a real thing. Lying or misrepresenting information on financial applications is illegal in most jurisdictions the world over. I have no trouble believing that a sub-agent of enough specificity would attempt to commit fraud in the pursuit of it's instructions.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#326

Post by 6510 »

WillAdams wrote: post #232
RupertSalt wrote: post #216

Earlier quoted context omitted.

How is a corporation "immortal"? What is the oldest corporation in the world? I mean, aside from churches and stuff. Corporations can die or be killed in numerous ways. Not many of them will live forever. Most will barely outlive a normal human's lifespan. By definition, since a corporation comprises a group of people, it could never outlive the members, should they all die at some point. Let us also draw a distincti…

Construction company okay? >Kongo Gumi, founded in 578 AD, is recognized as the oldest continuously operating company in the world, specializing in the construction of Buddhist temples.

Ah, so we should import Japanese people to run our companies.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#328

Post by nradov »

ajcp wrote: post #325
nradov wrote: post #297

Earlier quoted context omitted.

Illegal? Seriously? What specific crimes did they commit? Frankly I don't believe you. I think you're exaggerating. Let's see the logs. Put up or shut up.

Fraud is a real thing. Lying or misrepresenting information on financial applications is illegal in most jurisdictions the world over. I have no trouble believing that a sub-agent of enough specificity would attempt to commit fraud in the pursuit of it's instructions.

Do you believe allegations of criminal behavior based on zero reliable evidence? I hope you never end up on a jury.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#329

Post by InitialLastName »

willmarquis wrote: post #324

[flagged]

This is the "LLM as junior engineer (/support representative/whatever)" strategy. If you wouldn't equip a junior engineer to delete your entire user database, or a support representative to offer "100% off everything" discounts, you shouldn't equip the LLM to do it.

Re: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

#330

Post by FatherOfCurses »

blahgeek wrote: post #32

If human is at, say, 80%, it’s still a win to use AI agents to replace human workers, right? Similar to how we agree to use self driving cars as long as it has less incidents rate, instead of absolute safety

Oh yeah it's a blast for the human workers getting replaced.

It's also amazing for an economy predicated on consumer spending when no one has disposable income anymore.

Post reply on HN