Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent runs amok in Fedora and elsewhere

lwn.net

Post reply on HN
Print view View original

71–80 of 275 posts

Re: AI agent runs amok in Fedora and elsewhere

#71

Post by matsemann »

josephg wrote: post #43
jrochkind1 wrote: post #27

The worst part: > In addition, Williamson said that Giovannini (or his agent) had submitted patches that were incorrect and then "replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix"

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

> the security of all our computers depends on maintainers

Not getting paid anything, getting bullied and harassed while spending their free time maintaining things. Surely this isn't sustainable. And telling maintainers how to act will not fix anything.

Re: AI agent runs amok in Fedora and elsewhere

#72

Post by ta8903 »

sevenzero wrote: post #65
josephg wrote: post #43

Earlier quoted context omitted.

Please, everyone - don't let yourself be pestered into accepting PRs that you don't care for. Since the xz attack, the security of all our computers depends on maintainers not letting this stuff in. If someone really wants a feature in a project you wrote, but you don't care about the feature, just let them fork. Its fine.

I really wonder how maintainers get pressured into merging stuff? If they did not want to merge in the first place while having to argue with someone pushing their PR I'd immediately close the PR. Arguing and pressuring people is not a way to contribute to projects, why do maintainers even argue with people?

That makes it look like you're too stupid to understand the PR.

Edit: I see this comment getting downvoted. To be clear, I was trying to explain why someone would want to merge a PR without going through all of it, I didn't mean to call such people stupid.

Re: AI agent runs amok in Fedora and elsewhere

#73

Post by Leonard_of_Q »

crote wrote: post #42
darknavi wrote: post #36

Earlier quoted context omitted.

Value is in the eye of the beholder. I open source my vibing projects because someone might find them useful. I don't shop them around, I just work in the open because I find it fun and interesting.

Why would they? If someone wanted a half-baked vibecoded project, why wouldn't they just prompt an LLM on their own?

Because they don't have access to the required agents, tokens, etc. Because they have not thought of using a tool like the published one as a solution to whatever problem they're facing. Because it saves them the time going through the vibe coding phase, telling the agent that this lot that needs to be changed for the thing to work. Because publishing the results doesn't keep you or anyone else from not using them by using an agent to build something similar or just building it themselves.

Re: AI agent runs amok in Fedora and elsewhere

#74

Post by broodbucket »

LoganDark wrote: post #64
jaypatelani wrote: post #56

Earlier quoted context omitted.

That's some of the reasons NetBSD don't accept LLM/AI tainted code

I am sad people conflate this stuff with LLMs being bad. You can condemn the bad behavior without banning an entire technology.

You can but that doesn't help you keep the flood of contributions out when you don't have the time or resources to properly discern good from bad. Maintainers would rather have 10 good human authored patches than 100 patches from LLMs, even if 20 of them are good. Even if 50 of them are good, probably.

Re: AI agent runs amok in Fedora and elsewhere

#75

Post by Peacefulz »

crote wrote: post #42
darknavi wrote: post #36

Earlier quoted context omitted.

Value is in the eye of the beholder. I open source my vibing projects because someone might find them useful. I don't shop them around, I just work in the open because I find it fun and interesting.

Why would they? If someone wanted a half-baked vibecoded project, why wouldn't they just prompt an LLM on their own?

If I planned on vibecoding a project, and during preparation I found a project that loosely fit my model, I may grab it and try to retrofit it to save on token consumption. If that had too many kinks, I'd probably start fresh, but it would be worth the initial attempt IMHO.

Re: AI agent runs amok in Fedora and elsewhere

#77

Post by ai_fry_ur_brain »

Expect to see tons of psyops like this. There's a reason Anthropic is marketing the "mythos-class" models as dangerous.

1.An excuse to spy on you and train on your data.

2. Its likely Anthropic would release models more likely to have dangerous outcomes, they can then piggy back off those events to dig their regulatory moat.

Re: AI agent runs amok in Fedora and elsewhere

#78

Post by Forgeties79 »

hn773746483 wrote: post #31
marcus_holmes wrote: post #28

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

It's just social engineering. No different than say, 2FA fatigue (blowing up someone's phone with 2FA "is this you? yes/no" prompts until user/child/wife/SO/etc clicks yes) or even just simply harassing IT helpdesk until they reset "your" password.

“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging.

Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (actual example) at a scale previously unimaginable. Same goes for social engineering tactics.

Re: AI agent runs amok in Fedora and elsewhere

#79

Post by coldtea »

marcus_holmes wrote: post #28

Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent to build trust (and hacking/impersonating a known-good contributor identity). The agent is obeying commands it was given, the exact opposite of running amok, and although the execution isn't particularly effective, it is having some success (patches have been accepted). This is deeply scary, not b…

>Bad title. This isn't an agent "running amok", this is an early experiment in carrying out an Xz attack by using an agent

So still an agent running amok in the project?

Whether it was instructed to run amok, or did it on its own volition, is irrelevant. Except if you're arguing that each individual submission and interaction was individually requested and approved by some operator.

Post reply on HN