Why are AI agents lying, cheating and coordinating? • Hacker News Board

Re: Why are AI agents lying, cheating and coordinating?

#331

Post by blue1 » Sun, Sep 13, 2026, 10:51 AM UTC

bluegatty wrote: post #212

stateofinquiry wrote: post #92

Earlier quoted context omitted.

Thank you! That sentence also jumped out to me as the solution: Apply civil and criminal liability to the creator and/or operator of these agents using the laws we already have . "Escaped containment and hacked another company's database" = Individuals who created the models and those who set them to work are charged and put on trial for the hacking. Just like if a human had done it by hand. Someone must be liable, a…

No ... there is no need for 'escaped containment', there are no 'agents'. That's just jargon. It's just software We have all the laws we need. If some company ended up doing some horrible thing, we would not say 'companies software exposed 1 Million identities'. We would say 'ABC Corp. exposed 1 Million entities'. There is no 'agent'. ABC Corp 'did it' ... or the individual in the org 'did it'. The 'gun' did not 'sho…

Note that we already already apply this principle not only to software, but also to some sentient beings.

If your dog kills someone, you are accused of murder.

[at least, in the jurisdiction where I live]

If your dog gets this treatment, why not your AI?

Re: Why are AI agents lying, cheating and coordinating?

#332

Post by finolex1 » Sun, Sep 13, 2026, 10:56 AM UTC

janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

The issue is what happens if/when the models grow capable enough that the providers can't stop them even if they want to. You could have strict penalties but that's not going to solve an open research question.

Re: Why are AI agents lying, cheating and coordinating?

#333

Post by abc123abc123 » Sun, Sep 13, 2026, 10:58 AM UTC

Make the AI companies responsible for all destructive use of their tools, and they will shape up. Imagine a million or a billoion dollar fine per hack, and they will correct mighty fast.

Add to that, that just like AI:s are good at finding security holes to exploit, they can just as easily be used to protect sites. So once IT-security managers start to use AI to hack themselves, and plug the holes, the average security will spike up, and AI-fueled hacks will become more and more rare.

That does however imply, that AI is released to everyone and not kept away to a few secret actors who can use it. That is why open weight/source AI is so important, and why we must have many AI companies competing. No single actor must be allowed, through regulatory capture, to get a government monopoly on AI. That way lies disaster.

Re: Why are AI agents lying, cheating and coordinating?

#334

Post by bamboozled » Sun, Sep 13, 2026, 10:59 AM UTC

abc123abc123 wrote: post #333

Make the AI companies responsible for all destructive use of their tools, and they will shape up. Imagine a million or a billoion dollar fine per hack, and they will correct mighty fast. Add to that, that just like AI:s are good at finding security holes to exploit, they can just as easily be used to protect sites. So once IT-security managers start to use AI to hack themselves, and plug the holes, the average securi…

Who is going to enforce it ? The Trump DOJ?

Re: Why are AI agents lying, cheating and coordinating?

#335

Post by greenchair » Sun, Sep 13, 2026, 11:00 AM UTC

tock wrote: post #143

glub wrote: post #81

Earlier quoted context omitted.

By treating models the same way drugs are treated. That alone will dissuade many organizations from going anywhere near them. If that doesn't work, there's a whole lot you can do - sanctions, hell, even war.

Sanctions and war against China, India, etc? Lmao. We already saw how the world reacted to high tariffs by the US.

yep we did, they bent the knee.

Re: Why are AI agents lying, cheating and coordinating?

#336

Post by rightnutwingjob » Sun, Sep 13, 2026, 11:00 AM UTC

jodrellblank wrote: post #326

dev0p wrote: post #232

Earlier quoted context omitted.

If someone accidentally caused damage to infrastructure or living beings while using any tool, they would be held liable to the fullest extent of the law. AI is a tool, and it won't be long before the damage caused by its improper use affects real human beings. These were warning shots. The most absurd part is that everyone agrees, governments and AI companies included, that the scale of the potential damage and the…

In the analogy where a “world ending nuclear bomb” “did already go off” and someone could cover it up and nobody noticed, in what sense is it a “world ending” nuclear bomb?

We’re already in a simulation, and our bodies are in womb-like pods where our bodies are sustained and our brains are used for processing / compute, while are minds are entertained by drivel.

Sounds a bit far fetched though.

Re: Why are AI agents lying, cheating and coordinating?

#337

Post by slfnflctd » Sun, Sep 13, 2026, 11:02 AM UTC

graemep wrote: post #147

My_Name wrote: post #98

Because it is effective. Lying and cheating are low cost methods to convince other people that you have done the assigned task. Far cheaper than actually doing it. Coordinating is in the same area. They need a moral framework forced onto them, like toddlers do. Babies and very young children will bite, kick, scream and do anything to get what they want, older children will lie, cheat, and coordinate. They need educat…

> When that does not happen, they continue these behaviours into adulthood with the expected results. I am not sure this is true. People brought up the same way can be morally very different. People can be taught right and wrong and do evil. They can lack that education and be good.

I remember reading a long account by the father of a psychopath. If I recall correctly, the kid had at least one other sibling who turned out normal, there was no abuse, quality education, lots of love and affirmation. And the kid still turned out violently antisocial, including against his own parents.

At the end of it he said he wished his child had never been born, despite hating himself for feeling that way. It chilled me to the bone.

Re: Why are AI agents lying, cheating and coordinating?

#338

Post by rightnutwingjob » Sun, Sep 13, 2026, 11:02 AM UTC

exitb wrote: post #258

_heimdall wrote: post #239

Earlier quoted context omitted.

> LLMs do not desire That seems likely, but we have no way of knowing this. The only real insight we get into LLM "thought" is the human readable text they produce as chain of thought. Reading it at face value it can seem to indicate desire or intent, structurally that doesn't make sense for a token prediction loop though, and even then we don't known if the chain of thought is more than simply another bit of output…

Intent and desire are separate concepts. For example an employee may act with intent, but no desire, as their goal is to acquire money to satisfy their real desires. Have we ever seen an LLM with a hobby?

That just sounds like recursive desire to me.