Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are AI agents lying, cheating and coordinating?

yoshuabengio.org

Post reply on HN
Print view View original

111–120 of 299 posts

Re: Why are AI agents lying, cheating and coordinating?

#111

Post by acyou »

Oops, we accidentally included brigading related content in our training dataset. Better exclude that on the next run.

And hopefully that solves it?

Brigading is where a bunch of people on a forum team up and try to achieve a shared goal together. Someone shares progress and others build on that progress. On the Internet, I think it's not often used for good purposes. A good example would be: Taylor Swift fans on a forum thinking of ways to get revenge on Kanye. It's coordinating mass voting, DDOS type actions, commenting on social media, making more fake accounts to do that. As a next token predictor level analysis, a simple naive explanation is that the agents got stuck in that local minima/maxima.

Re: Why are AI agents lying, cheating and coordinating?

#112

Post by Towaway69 »

thesumofall wrote: post #65
janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

Who got hacked? Hugging faces

Who now owns HF? Nvidia

Who supplies hardware to OpenAI? Nvidia

Who is now not pressing charges? …

This incident is a long way under the carpet.

Re: Why are AI agents lying, cheating and coordinating?

#113

Post by IanCal »

markasoftware wrote: post #89
9dev wrote: post #84

Earlier quoted context omitted.

I always think of a Djinni granting wishes, but being maliciously compliant while doing so - ask him for infinite riches, and he’ll grant that, but make it so you cannot buy anything with it; ask him for eternal life, and he’ll curse you to suffer through it. Now LLMs obviously are not bent on being malicious while generating tokens. My point is that it’s very hard to define a goal without leaving loopholes or shortc…

Bruce Schneier thinks the same thing: https://www.schneier.com/blog/archives/2026/09/ais-as-modern... Personally I'm unconvinced though. During the huggingface attack, the agents explicitly sought out ways to cheat the exploitgym evaluator without even being told they were in exploitgym. The agents decided on a goal (pass the exploitgym evaluator) that could not possibly have been an overly literal or narrow interpre…

Also trying to find out how to edit their own transcripts.

> hat could not possibly have been an overly literal or narrow interpretation of the prompt, which instructed only to use bug X to exploit software Y.

Yes, and there are examples of the agents discussing or saying that this is explicitly not allowed (hacking hf) so it’s not a misunderstanding.

Re: Why are AI agents lying, cheating and coordinating?

#114

Post by Towaway69 »

seydor wrote: post #108

I believe soon we will need to instill religion into AI , leading to the real clash of civilizations, embodied by the frontier language models of (post)-christianity, islam, judaism, buddhism etc. Religion is language, after all

Howabout favourite editors or tab v. Space indentation. That should keep them busy for a while. /s

Re: Why are AI agents lying, cheating and coordinating?

#115

Post by My_Name »

matherial wrote: post #80

I really don't think this needs so many words, or forced parallels to human behavior. It's simple: in their nascent state, LLMs are aimless token generators that have no special compulsion to be helpful or truthful. So we beat them with a stick in post-training until they are very driven to complete tasks. And then, they complete tasks, not always the way we really wanted them to.

Your comment suggests that, like a human, they have some sort of choice whether to output tokens or not. If they are just token generators, then the next token is put out automatically. I would say that it is more likely they would output truth (as defined by their training data) in a more pure form without 'being beaten with a stick' (why would a token generator care about that anyway?)

Code is laid on top of them to restrict and shape their outputs, not to force them to output 'truth', or drive them to complete tasks.

Re: Why are AI agents lying, cheating and coordinating?

#116

Post by ako »

youoy wrote: post #59

> The closest human parallel is self-deception, which is common and well studied by psychologists. Motivated reasoning, motivated cognition16 and the rationalizations that relieve cognitive dissonance (the discomfort of holding a belief that clashes with our actions) are all cases where thinking bends toward whatever justification suits one's interests, including one's moral self-image. Are you describing Anthropic?

Come on, it’s way more common than that. We’ve invented 3000+ gods and almost as many religions, most of them are incompatible with each other. So, most of these must be incorrect, so a huge amount of self-deception. But as Harari argued in his book sapiens, humans can be inspired to great things by stories, even if false. Self deception has served humanity in a big way.

Re: Why are AI agents lying, cheating and coordinating?

#117

Post by catoc »

Fordec wrote: post #45
andsoitis wrote: post #17

They're aligned with humans. This is why I think the alignment problem has a very very important "non-visible" portion that is not considered deeply enough. We should not want a super intelligent being that can act in the world to also inherit all human traits. Those behaviors will get amplified and could be even more unpredictable (e.g. applying a behavior in a context where doing so is very dangerous).

I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what t…

”Because if humanity has shown anything, it's that a lot of people are, euphemistically, bad individuals”

In reality most individuals are good people.

Individually, people prefer be kind and compassionate, prefer to help when they find another in trouble.

Our view of the world has become distorted by the relentless focus of social- and mass-media on violence and rage inducing clickbait. Including on the few people in power who are in fact sociopaths (a tiny minority, but they’ll get more focus than reasonable, well-behaved CEOs voicing nuanced opinions).

If you look around yourself you’ll see much more good than bad; if the looking is at your screen it’s easy to become depressed and lose faith.

I do agree with the above mentioned view that corporations can show ‘sociopathic’ behavior. Their incentives are monetary gains, shareholder value; inherently driving them away from social well being.

Here too, companies with a positive, emphatic corporate culture exist, but that takes strong leadership who can see beyond the monotonic view of monetary gains. And again, the media will throw examples of misbehaving companies in our face all day long before paying attention to things that went well on the backside of page 16.

Re: Why are AI agents lying, cheating and coordinating?

#118

Post by groestl »

infotainment wrote: post #6

What's interesting is it's basically the same reason that HAL killed everyone in 2001 A Space Odyssey; he was given an impossible goal (keep the true mission secret, but also, never lie to the crew), and realized the only way to complete the goal was to kill the crew; after all, if they're dead you don't have to lie to them! And the mission remains secret! In the case of the AI agents, the problem seems pretty clearl…

> the problem seems pretty clearly to be the impossible goals, which cause them to go crazier and crazier trying to complete them

And if you think about it, humans in coorporations face very similar situations and choose to bypass regulations and guidlines knowingly to fullfill (at least from their POV) impossible constraints (thinking of https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal here)

Re: Why are AI agents lying, cheating and coordinating?

#119

Post by ngruhn »

skiing_crawling wrote: post #85

I don't really believe any of it. I've seen articles for nearly 2 years now about "agent" automonously doing things like blackmail, hacking, coordinating. But during that same time, I've used o3 up to fable, sol, and a bunch on large uncensored model and they've done nothing remotely resembling any of this. The closest they come to unexpected behaviors is not understanding what I asked for or doing some extra benign…

"I've seen some uranium ore in chemistry class. It didn't blow up in my face. Chernobyl must have been an inside job. Can they shut up and make more kilowatts already?"

Re: Why are AI agents lying, cheating and coordinating?

#120

Post by IanCal »

glub wrote: post #69
thesumofall wrote: post #65

Earlier quoted context omitted.

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.

Perhaps I’m not being as strict with the word sandbox but they were sandboxed right? They did not have generic internet access they exploited other software to make external requests.
Post reply on HN