Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

Post reply on HN
Print view View original

181–190 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#181

Post by sjamaan »

fc417fc802 wrote: post #104
jacobolus wrote: post #98

Earlier quoted context omitted.

[flagged]

[flagged]

> Regardless of your political views a tool is a tool at the end of the day. Attempting to anthropomorphize a category of objects in order to shift blame all for the sake of furthering an agenda is plainly bad faith behavior.

Guns are literally made for killing people. That's their only reason for existence. They are a weapon. This makes them qualitatively different from cars, which only incidentally kill people (and the vast majority of time, not on purpose).

To me, trying to equate deaths caused by purpose-made killing tools with those caused by generic tools is arguing in bad faith.

Re: AI agent runs amok in Fedora and elsewhere

#182

Post by haspok »

marcus_holmes wrote: post #144
coldtea wrote: post #79

Earlier quoted context omitted.

> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

"Amok" means "out of control" or "uncontrolled" [0][1] The agent was under control, as far as we can tell, and obeying its instructions. This is important for two reasons: 1. There are all the tropes of AI becoming uncontrolled and destroying humanity. Writing bad headlines around AI "running amok" feeds this. We should not be talking about this because it's not actually a problem. 2. It ignores, or overwrites, the m…

Certainly it might have been out of control of its original owner, perhaps due to a prompt injection attack. If I start a completely benign agent, but someone injects malicious instructions to it, would you still not say "the agent runs amok"?...

Re: AI agent runs amok in Fedora and elsewhere

#183

Post by mfru »

marcus_holmes wrote: post #144
coldtea wrote: post #79

Earlier quoted context omitted.

> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

"Amok" means "out of control" or "uncontrolled" [0][1] The agent was under control, as far as we can tell, and obeying its instructions. This is important for two reasons: 1. There are all the tropes of AI becoming uncontrolled and destroying humanity. Writing bad headlines around AI "running amok" feeds this. We should not be talking about this because it's not actually a problem. 2. It ignores, or overwrites, the m…

The web of trust finally becomes necessary and thus useful.

GNU was onto something apparently

Re: AI agent runs amok in Fedora and elsewhere

#184

Post by account42 »

jpalomaki wrote: post #139

Do we need to bring Keybase[1] "back"? The original idea, mapping your social media presence to certain encryption keys. In the future it will be increasingly difficult to prove in online context that you are not a bot. Being able to show that your social media (HN, GitHub, etc) presence goes way back would be an option. [1] https://en.wikipedia.org/wiki/Keybase

But the AI actions are already associated with a "real" pre-existing account in TFA, that didn't stop anything.

Re: AI agent runs amok in Fedora and elsewhere

#185

Post by m4rtink »

coldtea wrote: post #85
sevenzero wrote: post #65

Earlier quoted context omitted.

I really wonder how maintainers get pressured into merging stuff? If they did not want to merge in the first place while having to argue with someone pushing their PR I'd immediately close the PR. Arguing and pressuring people is not a way to contribute to projects, why do maintainers even argue with people?

> why do maintainers even argue with people Because they don't want to be seen like assholes, who just blindly dismiss PRs, and because they take the technical discussion about the PR in good faith.

On some of those PRs the AI agent (?) did not really pressure - it reacted promptly with changes and more plausible (hallucinated ?) technobabble why the PR is needed.

It can be quite hard to discern this behavior from a new contributor to the project that might be a domain expert on something you are not. Possibly with the exception of reacting far too quickly & enthusiastically compared to real people that might have a life.

Re: AI agent runs amok in Fedora and elsewhere

#186

Post by m4rtink »

crote wrote: post #39
literalAardvark wrote: post #38

Earlier quoted context omitted.

Nothing really stopping an agent from getting a key

The agent can't exactly show up to an in-person key signing party, can it? And how many people are both dedicated enough to go to key signing parties and stupid enough to let an agent act without supervision in the name of their real-world identity?

In this case the nathan-bot was also still on a plausible side - all the PRs looked kinda trivial & there were not outright rejections that would be a red flag for a maintainer checking the GitHub account activity during PR review.

Mucking with Bugzilla & reassigning bugs especially is what seems to have led to the discovery, rather than spotting an accumulation of nonsensical PRs or other behavior related to code unmasking the bot.

Re: AI agent runs amok in Fedora and elsewhere

#187

Post by bawolff »

yeodev wrote: post #122
bawolff wrote: post #66

> replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix In open source projects i participate in, "overwhelming" the maintainer gets you banned. It doesn't get your patches blindly merged. In some ways i find this one of the most shocking parts of the story.

As a "new" maintainer myself - how do you decide when to ban someone? I sometimes feel overwhelmed and I can feel a big uptick in huge PRs with huge LLM written descriptions but often I also don't want to be an asshole to my community & reject all their changes.

> but often I also don't want to be an asshole to my community & reject all their changes.

I know its difficult, and i have no easy answers. I'm bad at it too. But sometimes saying no is the most valuable thing you can do as a maintainer.

That said, i think banning is about behaviour not the quality of the patch. Everyone writes a bad patch now and then, that is not a real issue. If there is an issue with a patch, and the contributor pushes back so hard you feel like changing your mind (not from logic but because you feel beaten down) - that is unacceptable behaviour and should not be tolerated from a contributor, even if they are otherwise a valuable contributor.

Re: AI agent runs amok in Fedora and elsewhere

#190

Post by josephg »

dotancohen wrote: post #110
josephg wrote: post #89

Earlier quoted context omitted.

> telling maintainers how to act will not fix anything. I'm just saying its ok to ignore overly enthusiastic contributors and tell them to just fork your project. I think this does help, actually. In my early days of maintaining opensource software I felt burdened by open PRs - like I was letting someone down by ignoring their work. "Its ok, let them do whatever in their own fork" is advice I wish someone had given m…

> I'm just saying its ok to ignore overly enthusiastic contributors and tell them to just fork your project. I propose the phrasing "fork off".

A maintainer recently told me to “Fork baby, fork!” in response to a large patch set.

I was delighted.

Post reply on HN