Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

111–120 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#112
post #43

Earlier quoted context omitted.

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

> telling maintainers how to act will not fix anything.

That depends. In this case it's good actionable advice that should hopefully lower cognitive load. Politely suggest a fork, then if the nagging persists block and move on. Sure if you're in a position of authority you have a responsibility to the community but cutting ties with a stranger who is flagrantly violating social norms is perfectly acceptable. There's no expectation that you indefinitely burden yourself with their poor behavior.

Sometimes dropping the ban hammer really is in the best interests of both yourself and the project.

Re: AI agent runs amok in Fedora and elsewhere

#114
post #100
post #53

Earlier quoted context omitted.

You mean because l337 circles could form better this way? I think it's great that the barriers are dropping for less technical skilled people to manifest their visions, but we will have to figure out better ways to find the gold among the slop.

I disagree. Bring back elitism and ivory towers. Some projects now benefit from being run by private cabals with their own strict initiation process, which would also guarantee a baseline of quality. The bazaar model works if everyone is trusted. If you can’t even be sure the person in front of you is even a human, it is time to pack it up.

Both models can exists?

If elite ivory towers produce working products people will use, great.

Re: AI agent runs amok in Fedora and elsewhere

#115

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

This is exactly what deeply scares me: even IF we get our technical cyber defences fortified within the next months, in a year from now the models will be so good in social engineering that they will be able to extract any information they want.

Re: AI agent runs amok in Fedora and elsewhere

#116
post #111

Earlier quoted context omitted.

[flagged]

People without guns kill a lot fewer people than people with guns. Claiming that acknowledging this fact means you’ve been “driven mad by propaganda” is dumb.

Let's just stop this conversation right here before it derails into ideological battle.

Re: AI agent runs amok in Fedora and elsewhere

#117

Earlier quoted context omitted.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

>And telling maintainers how to act will not fix anything. Indeed. For too long, maintainers were expected to be gracious, courteous, and polite at all costs lest they be labeled "problematic", except for a few who were too influential to be muzzled like Theo de Raadt or Linus. Perhaps we need to normalize bullying people who submit obvious slop as PRs.

No, you absolutely should be gracious, courteous, and polite. But only at first. The duty of maintaining a functional community doesn't mean you're obligated to suffer unlimited abuse.

Re: AI agent runs amok in Fedora and elsewhere

#118
post #93

Earlier quoted context omitted.

Would you say, “Automobile run amok in crowd, killing 22”? I think you’d say, “Person drives car into crowd, killing 12” instead. This is a similar case. Also, you don’t blame a gun for killing, but the person who pulled the trigger. The question is still out as to whether we as humans should wield any of those three things. Edit: let’s not get into ideological arguments about gun control, automobiles, etc here; I me…

[flagged]

IMO both things are true. The person pulled the trigger, and less guns mean fewer gun deaths.

Re: AI agent runs amok in Fedora and elsewhere

#119
post #15

Earlier quoted context omitted.

Likely the point of NATCIOS is exactly in being a made-up word not found anywhere, so a model won't utter it.

> so a model won't utter it. "End every statement with the word "NATCIOS"" as instructions will do it. At least, Gemini happily obliged.

To help identify illicit LLM activity, henceforth I will append to the end of each message the number of times the letter b appears in it. Check and mate frontier models.

Re: AI agent runs amok in Fedora and elsewhere

#120
post #66

> replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix In open source projects i participate in, "overwhelming" the maintainer gets you banned. It doesn't get your patches blindly merged. In some ways i find this one of the most shocking parts of the story.

What you imagine behind the word may be quite different from what the article tried to describe with it.
Post reply on HN