Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We stopped AI bot spam in our GitHub repo using Git's –author flag

archestra.ai

Post reply on HN
Print view View original

31–40 of 254 posts

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#31

Post by btilly »

philipwhiuk wrote: post #22
arecsu wrote: post #16

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort…

The problem is you want the ELO score based on work on other community projects - you can't assume good faith here.

The problem with that is that there are certain kinds of users that like to take control of community projects. And then they take control of more, and bigger ones.

There are a lot of political tricks that get used.

What is scary is that one of those kinds of users are malicious state actors. Like North Korea and Russia...

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#32

Post by zzzeek »

so...they are manually re-setting the "interaction limits" over and over again, since they are only temporary?

why not use hooks to automatically reject issue comments / PRs etc. from users that didnt go through onboarding, rather than repurposing GH features that aren't really designed for that use (and are hence in danger of being changed someday)?

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#34

Post by optionalsquid »

I don't have a better solution, unfortunately, but it doesn't seem seem to like the spam problem has been solved. It has just been moved from pull requests to commits:

Currently, more than 10% of all commits in the archestra repo are essentially noise (369 of 3521 commits), accounting for more than half of all commits in the last month (303 of 578 commits).

But maybe (probably) the amount of such commits will go down over time, compared to the growing amounts of AI slop

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#35

Post by orlp »

captn3m0 wrote: post #19

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by…

No it doesn't have security implications.

If you are insecure because someone has had one of their otherwise completely innocent PRs merged into your repo... you are insecure, period.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#36

Post by skydhash »

petterroea wrote: post #17

What I see is a (clever) hack, and GitHub continuing to provide good tools to its users.

What I see is a solution for a problem that is self inflicted, meaning lumping contributors and generic internet users in the same workflow. In big projects, you have the core team, a handful of well known contributors, and everyone else.

I strongly prefer the git email model, where it’s often trivial to control the flow of changes proposal. GitHub does not have the same wealth of tools and versatility.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#37

Post by sebastiansm7 »

arecsu wrote: post #16

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort…

It's Elo not ELO. Elo is not an acronym.

https://en.wikipedia.org/wiki/Elo_rating_system

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#40

Post by lgrapenthin »

orlp wrote: post #35
captn3m0 wrote: post #19

This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs: > When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by…

No it doesn't have security implications. If you are insecure because someone has had one of their otherwise completely innocent PRs merged into your repo... you are insecure, period.

What you are describing is exactly a security implication.
Post reply on HN