Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

Post reply on HN
Print view View original

81–90 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#81

Post by coldtea »

Forgeties79 wrote: post #78
hn773746483 wrote: post #31

Earlier quoted context omitted.

It's just social engineering. No different than say, 2FA fatigue (blowing up someone's phone with 2FA "is this you? yes/no" prompts until user/child/wife/SO/etc clicks yes) or even just simply harassing IT helpdesk until they reset "your" password.

“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging. Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (a…

Yes. It's as if some people can't understand anything becoming a new huge problem unless that problem didn't exist at all before.

Re: AI agent runs amok in Fedora and elsewhere

#82

Post by coldtea »

LoganDark wrote: post #64
jaypatelani wrote: post #56

Earlier quoted context omitted.

That's some of the reasons NetBSD don't accept LLM/AI tainted code

I am sad people conflate this stuff with LLMs being bad. You can condemn the bad behavior without banning an entire technology.

But banning an entire technology is even better, as the potential for abuse and bad behavior is now scaled 1,000,000 times over.

Re: AI agent runs amok in Fedora and elsewhere

#83

Post by phoronixrly »

Leonard_of_Q wrote: post #76

There's a clear solution to the danger posed to free software projects by accepting hostile submissions but it probably is not one that maintainers want to hear: they can use an agent to check submissions for nefarious patterns. Sometimes you fight fire with fire.

And sometimes you fight this by disabling PRs in Github, and do not put more water into LLM providers' wheel.

Re: AI agent runs amok in Fedora and elsewhere

#84

Post by Forgeties79 »

coldtea wrote: post #81
Forgeties79 wrote: post #78

Earlier quoted context omitted.

“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging. Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (a…

Yes. It's as if some people can't understand anything becoming a new huge problem unless that problem didn't exist at all before.

At this point I just assume half of them are not saying it in good faith or at least with any real consideration. They just want to hand wave away whoever is critiquing their tools.

Re: AI agent runs amok in Fedora and elsewhere

#85

Post by coldtea »

sevenzero wrote: post #65
josephg wrote: post #43

Earlier quoted context omitted.

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

I really wonder how maintainers get pressured into merging stuff? If they did not want to merge in the first place while having to argue with someone pushing their PR I'd immediately close the PR. Arguing and pressuring people is not a way to contribute to projects, why do maintainers even argue with people?

>why do maintainers even argue with people

Because they don't want to be seen like assholes, who just blindly dismiss PRs, and because they take the technical discussion about the PR in good faith.

Re: AI agent runs amok in Fedora and elsewhere

#86

Post by LoganDark »

broodbucket wrote: post #74
LoganDark wrote: post #64

Earlier quoted context omitted.

I am sad people conflate this stuff with LLMs being bad. You can condemn the bad behavior without banning an entire technology.

You can but that doesn't help you keep the flood of contributions out when you don't have the time or resources to properly discern good from bad. Maintainers would rather have 10 good human authored patches than 100 patches from LLMs, even if 20 of them are good. Even if 50 of them are good, probably.

As if a rule against LLMs actually stops those sorts of spam contributions.

The only thing it does is filter good contributors out, while you still have to deal with the bad ones.

Re: AI agent runs amok in Fedora and elsewhere

#87

Post by stackghost »

matsemann wrote: post #71
josephg wrote: post #43

Earlier quoted context omitted.

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

>And telling maintainers how to act will not fix anything.

Indeed. For too long, maintainers were expected to be gracious, courteous, and polite at all costs lest they be labeled "problematic", except for a few who were too influential to be muzzled like Theo de Raadt or Linus.

Perhaps we need to normalize bullying people who submit obvious slop as PRs.

Re: AI agent runs amok in Fedora and elsewhere

#88

Post by rohitsriram »

The scariest part isn't the bad patches, it's that an agent overwhelmed a maintainer into merging something they didn't want to merge. That's not a technical attack, that's exhaustion being weaponized. Maintainers are already stretched thin and now the volume of confident-sounding noise is infinite and free. The attack surface was always human attention, not code review.

Re: AI agent runs amok in Fedora and elsewhere

#89

Post by josephg »

matsemann wrote: post #71
josephg wrote: post #43

Earlier quoted context omitted.

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

> telling maintainers how to act will not fix anything.

I'm just saying its ok to ignore overly enthusiastic contributors and tell them to just fork your project.

I think this does help, actually. In my early days of maintaining opensource software I felt burdened by open PRs - like I was letting someone down by ignoring their work. "Its ok, let them do whatever in their own fork" is advice I wish someone had given me.

Re: AI agent runs amok in Fedora and elsewhere

#90

Post by resonious »

coldtea wrote: post #79
marcus_holmes wrote: post #28

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

I think the point is that the title makes it sound like people lost control of the agent when really they're in full control.
Post reply on HN