Earlier quoted context omitted.
[flagged]
[flagged]
[deleted]
AI agent runs amok in Fedora and elsewhere
151–160 of 275 posts
Re: AI agent runs amok in Fedora and elsewhere
#152Earlier quoted context omitted.
[flagged]
People without guns kill a lot fewer people than people with guns. Claiming that acknowledging this fact means you’ve been “driven mad by propaganda” is dumb.
This is not true; there are quite a few people with guns who have never killed anyone, and quite a few people without guns who found a way to kill someone anyway. Poison, knives, hammers, rocks, windows, their bare hands. You name it someone has killed someone with it.
Re: AI agent runs amok in Fedora and elsewhere
#153Earlier quoted context omitted.
As a "new" maintainer myself - how do you decide when to ban someone? I sometimes feel overwhelmed and I can feel a big uptick in huge PRs with huge LLM written descriptions but often I also don't want to be an asshole to my community & reject all their changes.
Remove the human element. Yes, someone spent time fixing a bug. If the fix doesn't look like it makes sense on its own, do not merge it. If the author tries to convince you that it's a good fix, it's an immediate no. A good fix (which is the only acceptable fix in open-source software), is one that speaks for itself.
Well, I dunno. Sometimes the fix speaks for itself but the other party is as dumb as a box of rocks and doesn’t understand. It can be hard to tell the difference.
Re: AI agent runs amok in Fedora and elsewhere
#154[flagged]
Re: AI agent runs amok in Fedora and elsewhere
#155Earlier quoted context omitted.
From the article: > "So not saying this was it, but an AI agent automated attempt at a Xz like compromise might really look very similar what we have just seen here." Without identifying and interviewing the attacker we can't confirm that's what they intended, and there's a possibility that it was just incompetence/ignorance/whatever, but we should probably treat it as an attempted attack even if it wasn't.
We should treat it as attempted attack in the sense of preparing for the next one, but I don't see why we should call it "attack" without any evidence
We can call it an attack because the operator is responsible for the automation no matter what it does.
Re: AI agent runs amok in Fedora and elsewhere
#156Earlier quoted context omitted.
It's just social engineering. No different than say, 2FA fatigue (blowing up someone's phone with 2FA "is this you? yes/no" prompts until user/child/wife/SO/etc clicks yes) or even just simply harassing IT helpdesk until they reset "your" password.
It's scalable, personalizable social engineering. I think that makes it a lot more dangerous.
Yes but not free either. Spam works because it scales and even though 0.0000001% only might fall for it, it's still "worth" it. Here it might be 0.0001% instead but it's a lot more expensive, even with subsidized tokens, to do.
So it's interesting, feasible, but it's probably not as broad impact as the scariest scenario leads out to be.
Also I imagine that once exposed it becomes a well known pattern. Some will still fall from it but I imagine once it's been done few times it becomes even costlier.
The fact that Xz is mentioned and most of us know right away what it means show that we collectively learn.
Re: AI agent runs amok in Fedora and elsewhere
#157Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…
> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.
No, you're still anthropomorphizing an algorithm. Responsibility lies with the operator.
Re: AI agent runs amok in Fedora and elsewhere
#158Earlier quoted context omitted.
It's just social engineering. No different than say, 2FA fatigue (blowing up someone's phone with 2FA "is this you? yes/no" prompts until user/child/wife/SO/etc clicks yes) or even just simply harassing IT helpdesk until they reset "your" password.
“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging. Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (a…
True but it's an arm race.
Only mentioning that it feasible or even has been done few times mean that people who care will act accordingly. It doesn't remove the problem but it makes it radically less effective already by just being aware of it.
Re: AI agent runs amok in Fedora and elsewhere
#159Earlier quoted context omitted.
> Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent So still an agent running amok in the project? Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.
Would you say, “Automobile run amok in crowd, killing 22”? I think you’d say, “Person drives car into crowd, killing 12” instead. This is a similar case. Also, you don’t blame a gun for killing, but the person who pulled the trigger. The question is still out as to whether we as humans should wield any of those three things. Edit: let’s not get into ideological arguments about gun control, automobiles, etc here; I me…
Unfortunately the news commonly do put the automobile as the subject when the driver is of a class politically protected from blame. Just like with people anthropomorphizing AI, it serves to deflect blame from the real culprit.
Re: AI agent runs amok in Fedora and elsewhere
#160Earlier quoted context omitted.
Let's just stop this conversation right here before it derails into ideological battle.
No I think we should definitely find a creative way to drag at least abortion and freedom of speech into this "conversation". Fight fire with fire so to speak.
Well technically killing someone is just a really late abortion.