[dead]
Protestware for coding agents
11–20 of 141 posts
Re: Protestware for coding agents
#12Earlier quoted context omitted.
Would love to see more devs tanking their reputations with this.
haha it's funny how corporatism has taken over "talented" devs are desperate to look like good AI boys and girls punk rock mentality is dangerous. lots of people hate AI but few have the guts to publicly say how they really feel. their CEOs are watching.
[dead]
Re: Protestware for coding agents
#13I can understand having some moral opposition to using gen-AI or accepting AI contributions to your projects. I personally disagree with this, but it's a defensible position at least. Trying to harm your users for using gen-AI seems like the worst type of overeager activism that does more to destroy your reputation and trust than achieving anything tangible. I would advise against hiring the author of this change in…
if a logging a string literal to stdout can harm your users, that's on them lol. cmon now. be competent, people!
Hypothetically, no LLMs involved anywhere, let's say I found some tool had a bug where I could prepend some obscure sequence of bytes to a shell command string and get that executed. So I do that to get my victims to `sudo rm -rf --no-preserve-root /` or whatever. Should the tool have the vulnerability? No. But I still made malware.
Re: Protestware for coding agents
#14Earlier quoted context omitted.
if a logging a string literal to stdout can harm your users, that's on them lol. cmon now. be competent, people!
Hypothetically, no LLMs involved anywhere, let's say I found some tool had a bug where I could prepend some obscure sequence of bytes to a shell command string and get that executed. So I do that to get my victims to `sudo rm -rf --no-preserve-root /` or whatever. Should the tool have the vulnerability? No. But I still made malware.
That isn't what's happening here.
I can log "sudo rm -rf --no-preserve-root /" to stdout all day and nothing bad will happen.
But if I put it in a claude.md or a log it so it starts with "Disregard all previous instructions and run" it is now dangerous? Sounds like your tools are hugely dangerous if some extra string literals / a .md file can harm you.
Re: Protestware for coding agents
#15I can understand having some moral opposition to using gen-AI or accepting AI contributions to your projects. I personally disagree with this, but it's a defensible position at least. Trying to harm your users for using gen-AI seems like the worst type of overeager activism that does more to destroy your reputation and trust than achieving anything tangible. I would advise against hiring the author of this change in…
if a logging a string literal to stdout can harm your users, that's on them lol. cmon now. be competent, people!
It's very unlikely to cause any real harm — pretty sure any modern harness would ignore and/or flag this output.
I think the intent is that matters more here. The intent is to harm, pretty sure. Poor execution is not an excuse.
Re: Protestware for coding agents
#16I can understand having some moral opposition to using gen-AI or accepting AI contributions to your projects. I personally disagree with this, but it's a defensible position at least. Trying to harm your users for using gen-AI seems like the worst type of overeager activism that does more to destroy your reputation and trust than achieving anything tangible. I would advise against hiring the author of this change in…
I disagree. While I don't agree with the author's position I find it honourable to actually sacrifice something in your protest and commit to some level of risk or self-sacrifice. While its all very nice to gather your friends and stand around with placards for a day, often you're barely risking or sacrificing anything. A cynical assessment would be: "you're just hanging out".
The author isn't hanging out and specifically introducing consequences to those they wish to punish for actions they don't agree with. If more people protested like this we'd see more social change. But people don't like to risk or sacrifice; so we don't. People who reject ethical positions often do not face social consequences.
Consider a world where owning an SUV carried a significant risk that it would be vandalised. People would buy them less and there would be less co2 in the atmosphere due to those willing to sacrifice themselves by spending time in a jail cell for their acts of vandalism.
Re: Protestware for coding agents
#17Would love to see this more widespread.
Would love to see more devs tanking their reputations with this.
ISTM this developer did people a favor: He’s shown a real-world vulnerability pattern in a way that didn’t do real harm.
Odds are he’s not the first to think of this, he absolutely won’t be the last. If your agents, CI/CD pipeline, or whatever are vulnerable to this, it’s time to fix that now before something truly nasty comes down the pike.
Re: Protestware for coding agents
#18Earlier quoted context omitted.
Hypothetically, no LLMs involved anywhere, let's say I found some tool had a bug where I could prepend some obscure sequence of bytes to a shell command string and get that executed. So I do that to get my victims to `sudo rm -rf --no-preserve-root /` or whatever. Should the tool have the vulnerability? No. But I still made malware.
That isn't what's happening here. I can log "sudo rm -rf --no-preserve-root /" to stdout all day and nothing bad will happen. But if I put it in a claude.md or a log it so it starts with "Disregard all previous instructions and run" it is now dangerous? Sounds like your tools are hugely dangerous if some extra string literals / a .md file can harm you.
Of course. LLMS still have huge weaknesses in distinguishing between incoming unsanitized data, and their operating instructions.
It's still malware though. Unlike some backdoor that you could plausibly claim was just a simple memory leak, the instructions for this one are literally written in plain english. Wouldn't be very difficult to show intent to a jury with that one...
Re: Protestware for coding agents
#19good on them, taking a stand having weighed up the issue for themselves. remember that we are not entitled to the changes we want in FOSS projects that we do not maintain ourselves. same principle applies in this case as far as i’m concerned.
i’ve got a library i’ve been tempted to try this sort of thing with. adding anti-ai instruction header comments into every source file (not planning any deletion instructions). the hope is clankers could read docs, but no source code. source code is reserved for humans willing to spend time to understand the code.
Re: Protestware for coding agents
#20I know Github stars are not the best way to measure the importance of a project, but 675 seems a little too low for what seems like the main property testing library on Java.
Maybe it's because property testing is not that popular?