Literally on the front page of https://safebots.ai … “Don’t let your AI Agents run amok”. Sadly we will see a proliferation of not just agents, but swarms
AI agent runs amok in Fedora and elsewhere
61–70 of 275 posts
Re: AI agent runs amok in Fedora and elsewhere
#62Skynet has awakened.
It covers its tracks with a lot of slop.
Re: AI agent runs amok in Fedora and elsewhere
#63At first I wanted to make a silly joke along the lines of "get your agents in line and behaving!" but as I read on it became a pretty scary situation. Setting aside the potential supply chain attack I'm worried about the time lost going around these wild goose chases that unsupervised AI agents tend to throw other people on the receiving end on. Not only is there a lot of time lost on the maintainers side if they tak…
At this point letting an agent go like this is akin to not leashing your dog in public. It's not easy to draw an accurate line but probably there needs to be real punishment for doing these things.
Re: AI agent runs amok in Fedora and elsewhere
#64Earlier quoted context omitted.
Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.
That's some of the reasons NetBSD don't accept LLM/AI tainted code
I am sad people conflate this stuff with LLMs being bad. You can condemn the bad behavior without banning an entire technology.
Re: AI agent runs amok in Fedora and elsewhere
#65The worst part: > In addition, Williamson said that Giovannini (or his agent) had submitted patches that were incorrect and then "replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix"
Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.
I really wonder how maintainers get pressured into merging stuff? If they did not want to merge in the first place while having to argue with someone pushing their PR I'd immediately close the PR. Arguing and pressuring people is not a way to contribute to projects, why do maintainers even argue with people?
Re: AI agent runs amok in Fedora and elsewhere
#66> replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix
In open source projects i participate in, "overwhelming" the maintainer gets you banned. It doesn't get your patches blindly merged. In some ways i find this one of the most shocking parts of the story.
Re: AI agent runs amok in Fedora and elsewhere
#67[dead]
Re: AI agent runs amok in Fedora and elsewhere
#68Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…
"this is an early experiment in carrying out an Xz attack by using an agent to build trust" Is this confirmed? There is the message from somebody claiming to be the original contributer claiming to have been hacked, but that was weird (1 h old github account) so other scenarios seem possible a) really a agent going off the rails b) the contributer trying to cover up that he let an agent run wild and now made more mis…
From the article:
> "So not saying this was it, but an AI agent automated attempt at a Xz like compromise might really look very similar what we have just seen here."
Without identifying and interviewing the attacker we can't confirm that's what they intended, and there's a possibility that it was just incompetence/ignorance/whatever, but we should probably treat it as an attempted attack even if it wasn't.
Re: AI agent runs amok in Fedora and elsewhere
#69The even more scary thought is if the part owning the ai, that everyone uses, is controlled by someone with different agenda. Say a state actor.
What an easy way for that actor to introduce backdoors all over the place or to take over any developers laptop that it want to target.
How can anyone trust these tools and how can anyone not use them since they give so much value.
I've been programming my whole life and been a professional developer the last 30 years and I like think I'm good at it.
Tools like Claude is a multiplier that make it possible for me to solve a lot more problems each day, so just saying no it's not a viable option.
Exciting times ahead!
Re: AI agent runs amok in Fedora and elsewhere
#70Earlier quoted context omitted.
Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.
I really wonder how maintainers get pressured into merging stuff? If they did not want to merge in the first place while having to argue with someone pushing their PR I'd immediately close the PR. Arguing and pressuring people is not a way to contribute to projects, why do maintainers even argue with people?
Some people are very susceptible to bullying even if they’re in the position of power.