Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We stopped AI bot spam in our GitHub repo using Git's –author flag

archestra.ai

Post reply on HN
Print view View original

41–50 of 254 posts

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#41

Post by finseam »

ildari wrote: post #23
captn3m0 wrote: post #19

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by…

fair point! We believe "Require approval for all external contributors" should be a default setting, as you cannot trust anyone who is not a member of the organization

Interesting approach. We’ve seen similar spam/noise problems appear in financial workflow automation too — especially when AI-generated submissions scale faster than manual review processes.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#42

Post by cermicelli »

ildari wrote: post #23
captn3m0 wrote: post #19

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by…

fair point! We believe "Require approval for all external contributors" should be a default setting, as you cannot trust anyone who is not a member of the organization

you can't trust org members either I have seen projects have inter maintainer fallouts. In general trust doesn't exist.

If companies can screw you over and claim it's a mistake, there isn't much a person can do.

It's all about level's of trust, a maintainer going rogue is less likely, a past contributor going rogue more likely but not too much, a stranger with a typo pr merged even more likely but still, a complete stranger least trust worthy.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#43

Post by ZeWaka »

microtonal wrote: post #28
silverwind wrote: post #5

PR spam is a major problems for repo that run bounties. Maybe GitHub should temporarily block accounts from raising PRs if like 95%+ of them are getting rejected.

I feel like GitHub should have a system where you can give out tokens that are valid for e.g. 1 PR. If someone shows to engage in meaningful discussion and has a good idea to address an issue/feature, you initially give them one PR token. If the PR is of good quality, you can give them a few more, until they are contributors that can just create PRs as they like. A similar system would be nice for issues, though I'm…

My community does something vaguely similar, where you get credit for having bugfix PRs merged, and it's deducted when you get feature PRs merged.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#44

Post by stavros »

orlp wrote: post #35
captn3m0 wrote: post #19

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by…

No it doesn't have security implications. If you are insecure because someone has had one of their otherwise completely innocent PRs merged into your repo... you are insecure, period.

Security isn't a binary "secure/insecure". You can be more or less secure than something.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#45

Post by morkalork »

btilly wrote: post #27
arecsu wrote: post #16

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort…

ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat. Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating ot…

Reputation scores, review cartels. This all sounds familiar!

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#48

Post by chii »

btilly wrote: post #27
arecsu wrote: post #16

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort…

ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat. Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating ot…

fix this problem by make the rating value tied to some paid currency - a repo owner would have to pay for the PR, and that PR contributor will now have more currency than previously. In order to have said currency to pay, the repo owner would need to have contributed to another repo whose owner have currency.

The totality of someone's currency is their reputation.

Of course, now the decision becomes...who is the central currency issuer that creates it?

Post reply on HN