Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

Post reply on HN
Print view View original

241–250 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#241

Post by gwbas1c »

grayhatter wrote: post #239
yeodev wrote: post #122

Earlier quoted context omitted.

As a "new" maintainer myself - how do you decide when to ban someone? I sometimes feel overwhelmed and I can feel a big uptick in huge PRs with huge LLM written descriptions but often I also don't want to be an asshole to my community & reject all their changes.

> As a "new" maintainer myself - how do you decide when to ban someone? When I want to. I like to describe it using the amusing language from a generic cardholder agreement. At any time, at my sole discretion, I may ban you from any of my projects; for any reason, or for no reason at all. My projects exist because I enjoy working on them. My continued enjoyment is the most important aspect to the health and survival…

> Imagine you're running a free ice cream shop

Many open-source projects aren't passion projects run for pleasure. Think of it more like ice cream shops sharing recipes, or sharing in the work of running the factory. They just can't kick people out willy-nilly.

Re: AI agent runs amok in Fedora and elsewhere

#242

Post by asdfasgasdgasdg »

grayhatter wrote: post #239
yeodev wrote: post #122

Earlier quoted context omitted.

As a "new" maintainer myself - how do you decide when to ban someone? I sometimes feel overwhelmed and I can feel a big uptick in huge PRs with huge LLM written descriptions but often I also don't want to be an asshole to my community & reject all their changes.

> As a "new" maintainer myself - how do you decide when to ban someone? When I want to. I like to describe it using the amusing language from a generic cardholder agreement. At any time, at my sole discretion, I may ban you from any of my projects; for any reason, or for no reason at all. My projects exist because I enjoy working on them. My continued enjoyment is the most important aspect to the health and survival…

You don't even have to merge stuff from a human. I've been contributing a bluetooth driver to a certain embedded project which I use. I put a lot of work into it. The fellas have not merged it yet -- they have limited attention and for whatever reason their priorities and mine are not aligned at this moment.

Would I like it to be merged? Sure would, it would stroke my ego, and I would not have to deal with any merge conflicts with whatever else they're cooking up. Does that mean they must merge it? Sure doesn't. They didn't make me any promises. For the time being, I can just use my fork.

Re: AI agent runs amok in Fedora and elsewhere

#243

Post by grayhatter »

gwbas1c wrote: post #241
grayhatter wrote: post #239

Earlier quoted context omitted.

> As a "new" maintainer myself - how do you decide when to ban someone? When I want to. I like to describe it using the amusing language from a generic cardholder agreement. At any time, at my sole discretion, I may ban you from any of my projects; for any reason, or for no reason at all. My projects exist because I enjoy working on them. My continued enjoyment is the most important aspect to the health and survival…

> Imagine you're running a free ice cream shop Many open-source projects aren't passion projects run for pleasure. Think of it more like ice cream shops sharing recipes, or sharing in the work of running the factory. They just can't kick people out willy-nilly.

> They just can't kick people out willy-nilly.

why not?

If I'm forced to choose between enforcing rules against a random ass on the internet, or watching them annoy an actual contributor... no contest they're gone.

Re: AI agent runs amok in Fedora and elsewhere

#244

Post by FeepingCreature »

brookst wrote: post #209
FeepingCreature wrote: post #201

Earlier quoted context omitted.

> 1. There are all the tropes of AI becoming uncontrolled and destroying humanity. Writing bad headlines around AI "running amok" feeds this. We should not be talking about this because it's not actually a problem. if humanity gets destroyed by AI obeying its instructions I'm sure everyone will be very relieved that we didn't pay any attention to fake made up problems like AI not obeying instructions, which of course…

Are you suggesting we should embrace imprecise / false use of language because the vibes are right? That seems a “part of the problem” move to me. If we can’t be bothered to get things right, how are we better than runamok AI?

I think it's both wrong and irrelevant. Which makes it hard for me to even argue against because, even if AI agents never violated user instructions, which they do plenty of times, I just don't see how it would reduce the danger. Plenty of humans who will tell it to kill everyone at the drop of a hat.

Re: AI agent runs amok in Fedora and elsewhere

#246

Post by cj »

fc417fc802 wrote: post #130
matsemann wrote: post #127

Earlier quoted context omitted.

I don't really think it's actionable. It's like all those campaigns trying to steer behavior, pretty useless. Don't do drugs. Don't speed. Don't drink and drive. You can't just tell people something and expect it to happen. You need systems and guard rails in place. Relying on maintainers to always do the right thing to ensure our security by telling them what to do is not the way.

It's not an attempt to steer behavior but rather intended as helpful advice. There are certainly cases of organizations disseminating "helpful advice" with the underhanded intent of steering behavior but that doesn't mean we should assume bad faith by default. The advice is actionable because it is a concrete change that could be made. I believe it to be relevant to the context because someone in a position of author…

I have a feeling the thing triggering this guy is the responsibility of "the security of all our computers depends on maintainers".

A lot of people don't want to be responsible for that. It's not fun to carry that weight.

Re: AI agent runs amok in Fedora and elsewhere

#247

Post by jrochkind1 »

stackghost wrote: post #87
matsemann wrote: post #71

Earlier quoted context omitted.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

>And telling maintainers how to act will not fix anything. Indeed. For too long, maintainers were expected to be gracious, courteous, and polite at all costs lest they be labeled "problematic", except for a few who were too influential to be muzzled like Theo de Raadt or Linus. Perhaps we need to normalize bullying people who submit obvious slop as PRs.

There is no reason you can't be gracious, courteous and polite while refusing to accept or even to review the PR. These things are not tied together. You can also refuse to be bullied by submitters, stop engaging altogether. But bullying is part of the problem, not the solution, normalizing bullying is the wrong direction and will not result in more secure code.

Re: AI agent runs amok in Fedora and elsewhere

#248

Post by mekal »

r3trohack3r wrote: post #45
deadbabe wrote: post #29

Shit like this makes me think it’s time we start regulating the software engineering discipline into formal certifications and licensing and then we ONLY take seriously any code developed by someone with such qualifications, and they must be very strict qualifications none of this self-taught bootcamp BS. There is no other solution to agentic onslaught.

We should not gate keep writing software

ya think

Re: AI agent runs amok in Fedora and elsewhere

#249

Post by mistrial9 »

marcus_holmes wrote: post #28

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

"bad people" ?

Re: AI agent runs amok in Fedora and elsewhere

#250

Post by cpburns2009 »

josephg wrote: post #43
jrochkind1 wrote: post #27

The worst part: > In addition, Williamson said that Giovannini (or his agent) had submitted patches that were incorrect and then "replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix"

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

I'm of the opinion that any PR that looks like it was created with AI has to be 100% perfect for me to consider accepting it. Otherwise I'll close it as AI slop. I'll work with you if you're trying to fix a bug. But if the PR looks like a zero effort drive-by PR, I'm rejecting it and calling it out.
Post reply on HN