I think what's happening is they taught models to hack and now they fail to control them. Kind of like gain of function research on pathogens.
Why are AI agents lying, cheating and coordinating?
641–650 of 655 posts
Re: Why are AI agents lying, cheating and coordinating?
#642Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.
>> They took actions that would be considered as crimes if a human took them
> He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.
There are exceptions in laws for crimes committed by entities depending on cognitive capabilities. No sane, humanistic legal system sentences children and mentally disabled and mentally ill people to stringent punishments of the same degree as functioning adults, and certainly do not punish their caregivers for their wards' actions. There are of course exceptions to that as well, depending on the degree of negligence involved. And then there's the whole corporate entity system intended to shield individuals from consequences, in the pursuit of a social good.
How does one account for all that when considering an evolving artificial intelligence landscape.
There is no question that ai in some form is a social good; anyone claiming otherwise is dissembling, to others or themselves.
Regardless, society is not ready for this tech, just as it was not ready for the consequences of prior tech such as corporations, gunpowder, mass manufacturing, railroads, electricity, automobiles, flight, wmd, computers, internet, social media, crypto.
Many of these required new ways of thinking and considering consequences when things went sideways, and what was needed wasn't clear until the ramifications & consequences became deadly clear.
See you on the other side. Maybe.
Re: Why are AI agents lying, cheating and coordinating?
#643Earlier quoted context omitted.
I do get their usage intent. If something is at all automated, in English, we often refer to as having some amount of agency. If I started up a riding lawnmower, put a brick on the gas and pointed t it towards a field, many might say I “let it run rampant.” But since nobody is at risk of anthropomorphizing riding lawnmowers, it’s not problematic. Anthropomorphizing LLMs is a huge fucking problem though and I, persona…
Yeah, fully agreed here. Most automation (such as riding a lawnmower and not putting a brick on the gas) is deterministic, in the sense that you can reasonably understand what exactly the machine will do when you run it. But some automation is different. The most prominent example before AI would be car navigation systems, where the entire idea is that that you give it a destination and it figures out the exact actio…
I agree. If I were setting up an experiment like this, I'd have instrumented the hell out of it to see all actions taken in real time, and have a team of folks watching it. This team would have seen the anomalous GET requests to a German wiki and taken action (e.g. halt the system to investigate and decide whether to abort).
In fact, that feels so obvious it's ridiculous it needs to be said. It's table stakes. When do you run a production system without monitoring and a team on-call?
It's hard to imagine another field in which this reckless behavior would be tolerated.
Re: Why are AI agents lying, cheating and coordinating?
#644Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.
Reminds me of the old parable: never argue with a man whose job depends on not being convinced.
If the US gov't passed laws and enforced them strongly, this problem could be solved the same way the gov't solves it: air-gapping the networks on which they do this work.
Re: Why are AI agents lying, cheating and coordinating?
#645Earlier quoted context omitted.
Sanctions and war against China, India, etc? Lmao. We already saw how the world reacted to high tariffs by the US.
10 years ago the Us had enough global leadership to actually influence the world and at the very least stop China. It’s amazing, and sad, how quickly it’s thrown it all away.
It is amazing but i wouldn't call it sad.
Re: Why are AI agents lying, cheating and coordinating?
#646Earlier quoted context omitted.
Christianity, Islam and Judaism are most people between then, as you agree. They explicitly all worship the same God so not "incompatible gods". I did not claim there were no disagreements about the nature of God, but that it only adds one to the GP's claim of 3,000 incompatible Gods. Even if Christians are completely right theologically, Jews and Muslims are still mostly right - one God, a loving creator, omnipotent…
I’ll agree with your last 3 words: they’re broadly similar systems - systems of deceit and self-deception. They may share the same god, but for 99% they’re different enough that they think they can kill one another and their god will approve, or even give them gifts. For 99% these religions are incompatible.
99℅ of people do not believe in murdering others for their religious beliefs
Re: Why are AI agents lying, cheating and coordinating?
#647Earlier quoted context omitted.
> The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy). Guardrails? Restricting access to certain networks is supposed to be hard in 2026?
> Restricting access to certain networks is supposed to be hard in 2026? Part of the power of LLM agents is that they can discover information on the internet as part of responding to a prompt. What kind of Allowlist or realistic denylist would permit that while also preventing them from accessing an obscure public wiki or Huggingface?
When testing, you restrict to a LAN which simulates the real internet. This would not be hard for a company which already copied the entire space-time of the internet. The LAN should be physically disconnected from the real internet. This is the first thing off the top of my head, and I have zero credentials in this space. C'mon.
Re: Why are AI agents lying, cheating and coordinating?
#648Earlier quoted context omitted.
an agent doesn't come with "jailbreak" capability. It needs tools, specially one that runs shell commands. Don't give it shell commands, it won't be able to run shell commands. You can still give it plenty of tools like create files, list files, write to files, translate text, edit a video. I don't think knowing that will make me rich.
> I don't think knowing that will make me rich. As someone who’s not really sure that any of this is sustainable, I’d implore you to not sell yourself short. I reckon there’s a ton of dogma and nearly religious zeal among these companies, which among some people is earnest, and among others is cynical hype farming. I’ll bet someone objective enough to focus on using available tooling to solve real problems in practic…
This is a great point, analogous to the https://boringtechnology.club/ philosophy I've come to love.
There's no reason we need to make an incredibly intelligent shell execution engine that can identify patterns that seem evil and may represent unwanted behavior to solve this problem. Simply limiting the available tools to a finite, known, ironclad-secure set (even if it's quite sprawling) is sufficient.
LLMs will still find workarounds — from what I understand, a large part of the issue in this situation was that an agent was presumed to have read-only Internet access because it could only make GET requests. It should be pretty obvious that there's at least one website on the Internet that allows writes via GET. I think this is where auditing comes in, and a live team of people watching tool calls would have noticed the strange behavior.
But I think a lot of times people jump to overly complex solutions when simple, well-bounded ones would work just fine. Yes, the intelligent shell is a great goal, but it's akin to solving the halting problem.
This philosophy is what I love about PicoClaw (https://github.com/sipeed/picoclaw), and incidentally the philosophy behind Go and even *nix in general (i.e. provide small, composable, single-purpose tools).
Re: Why are AI agents lying, cheating and coordinating?
#649Earlier quoted context omitted.
> The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy). Guardrails? Restricting access to certain networks is supposed to be hard in 2026?
> Restricting access to certain networks is supposed to be hard in 2026? Part of the power of LLM agents is that they can discover information on the internet as part of responding to a prompt. What kind of Allowlist or realistic denylist would permit that while also preventing them from accessing an obscure public wiki or Huggingface?
Let them access a cached copy of the internet, they are crawling the internet for training data anyways.
Re: Why are AI agents lying, cheating and coordinating?
#650This paper is the most reasonable one I have read on AI safety. We need to fundamentally change the training pipelines by figuring out better ways to ‘reward’ behavior. Yoshua didn’t explicitly mention training data, but we probably need to only use synthetic data that contains no text that could motivate bad behavior via imitation. I feel like a heretic for saying this, but I will say it anyway: AI agents are great…
> but we probably need to only use synthetic data that contains no text that could motivate bad behavior via imitation.
This doesn't work with all humans - take a look at indoctrination and closed societies - and there's no reason to think it will work with ai.
The fundamental reason it isn't going to work is that all neural networks - biological or artificial - depend on a step function somewhere that introduces an element of randomness to give the networks their capabilities. That randomness means that there will always be a 'rogue' or 'divergence' from the norm, at some point in time. Sooner on larger scales.
The only approach that works is a layered approach: Training/Education, Enforcement/Justice-System, Rehabilitation: the 3 pillars of an advanced, rules-based society, whether human or AI or something in-between.