Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

Post reply on HN
Print view View original

101–110 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#101

Post by srdjanr »

ok_dad wrote: post #93
coldtea wrote: post #79

Earlier quoted context omitted.

> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

Would you say, “Automobile run amok in crowd, killing 22”? I think you’d say, “Person drives car into crowd, killing 12” instead. This is a similar case. Also, you don’t blame a gun for killing, but the person who pulled the trigger. The question is still out as to whether we as humans should wield any of those three things. Edit: let’s not get into ideological arguments about gun control, automobiles, etc here; I me…

There's a difference between the driver intentionally driving into crowd, and not intentionally but possibly still recklessly (drifting and losing control, falling asleep, etc). In those cases I would probably use "car hits the crowd", at least in my language

Re: AI agent runs amok in Fedora and elsewhere

#102

Post by [deleted] »

coldtea wrote: post #79
marcus_holmes wrote: post #28

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

[deleted]

Re: AI agent runs amok in Fedora and elsewhere

#103

Post by fc417fc802 »

tikkabhuna wrote: post #96
ok_dad wrote: post #93

Earlier quoted context omitted.

Would you say, “Automobile run amok in crowd, killing 22”? I think you’d say, “Person drives car into crowd, killing 12” instead. This is a similar case. Also, you don’t blame a gun for killing, but the person who pulled the trigger. The question is still out as to whether we as humans should wield any of those three things. Edit: let’s not get into ideological arguments about gun control, automobiles, etc here; I me…

Neither the automobile nor a gun can operate without a human. You could say “bull runs amok in a market” after it was released intentionally.

So the agent is exhibiting an unknown amount of autonomy thus we can't be certain whether "running amok" carries the correct connotation.

However that phrasing is also commonly used when a person or group wreaks havoc in a seemingly unpredictable manner. So I think the appropriateness comes down to how much chaos it has created and the level of apparent confusion on the ground.

Re: AI agent runs amok in Fedora and elsewhere

#104

Post by fc417fc802 »

jacobolus wrote: post #98
ok_dad wrote: post #93

Earlier quoted context omitted.

Would you say, “Automobile run amok in crowd, killing 22”? I think you’d say, “Person drives car into crowd, killing 12” instead. This is a similar case. Also, you don’t blame a gun for killing, but the person who pulled the trigger. The question is still out as to whether we as humans should wield any of those three things. Edit: let’s not get into ideological arguments about gun control, automobiles, etc here; I me…

[flagged]

[flagged]

Re: AI agent runs amok in Fedora and elsewhere

#105

Post by ezst »

Forgeties79 wrote: post #84
coldtea wrote: post #81

Earlier quoted context omitted.

Yes. It's as if some people can't understand anything becoming a new huge problem unless that problem didn't exist at all before.

At this point I just assume half of them are not saying it in good faith or at least with any real consideration. They just want to hand wave away whoever is critiquing their tools.

This, and/or the tendency in tech circles to "think in absolutes” (like in code, seeing things binary, ...) which is especially annoying in security-related discussions.

Re: AI agent runs amok in Fedora and elsewhere

#108

Post by LoganDark »

sph wrote: post #92
LoganDark wrote: post #64

Earlier quoted context omitted.

I am sad people conflate this stuff with LLMs being bad. You can condemn the bad behavior without banning an entire technology.

Technology doesn’t exist in a vacuum, you need the consider the possibility it will be used for evil and the effect that might result from that. Far too many people dismiss LLM risks with ‘oh, if people just stop being gullible/greedy/lazy everything will be fine’, as if that is a sensible proposition. In fact, LLMs proliferate in exactly because people are gullible, greedy and lazy and it’s easier to write a prompt…

I do consider the possibility it will be used for evil -- and then I ban evil.

Re: AI agent runs amok in Fedora and elsewhere

#110

Post by dotancohen »

josephg wrote: post #89
matsemann wrote: post #71

Earlier quoted context omitted.

> the security of all our computers depends on maintainers Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

> telling maintainers how to act will not fix anything. I'm just saying its ok to ignore overly enthusiastic contributors and tell them to just fork your project. I think this does help, actually. In my early days of maintaining opensource software I felt burdened by open PRs - like I was letting someone down by ignoring their work. "Its ok, let them do whatever in their own fork" is advice I wish someone had given m…

  > I'm just saying its ok to ignore overly enthusiastic contributors and tell them to just fork your project.
I propose the phrasing "fork off".
Post reply on HN