Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We stopped AI bot spam in our GitHub repo using Git's –author flag

archestra.ai

Post reply on HN
Print view View original

151–160 of 254 posts

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#151

Post by Terr_ »

drum55 wrote: post #93
infinitifall wrote: post #54

Is the solution to everything simply more catgirls [1]? Proof-of-work was, after all, about countering email spam. PR spam is but the latest in that long tradition. 1- https://anubis.techaro.lol

Proof of work doesn’t work here same as it doesn’t work for email. The effort to mint a valid PoW is always going to put the legitimate user at a disadvantage, whatever the implementation is. Someone with an incentive to spam will always be able to do it faster, more efficiently than you. You can’t submit a PR because your laptop is too slow? Rent some hash rate from someone, and now you’ve just made a system of payi…

> Someone with an incentive to spam will always be able to do it faster, more efficiently than you.

Sure, but looking at the cost to do it at scale is the wrong metric. I surely can't compete with a career spammer on emails-per-second or even emails-per-dollar, but I also don't need to.

It's more about the expected-value versus the cost. For example, my expected benefit from one email to my family is (while hard to quantify) hopefully much higher than a spammer's expected benefit of one spam email going out, which has a very small chance of leading to any amount of money. Attaching a CPU-churn cost per email is something I can ignore on my desktop, but they have to at least budget for it.

I'd also like to note that the win-condition isn't as extreme as making spam (or other "crimes") truly unprofitable, it just needs to be less profitable than other things the time/resources could be used for.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#152

Post by skrebbel »

rglullis wrote: post #71
skrebbel wrote: post #66

Earlier quoted context omitted.

Easily? You think the kind of people who think it makes sense to make bogus slop PRs are going to react reasonably to overburdened volunteer maintainers refusing to give them their US$10 back?

Yes. Once a PR is rejected, contact from that bot is blocked. No appeals.

This is never going to work. Sufficiently many of these people are going to find maintainers' home addresses and send them death threats and the likes. If you see how badly some people flip out just because their PR is rejected, it's going to be much much worse if their PR is rejected and their money is taken.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#153

Post by 20k »

jmuguy wrote: post #143
halapro wrote: post #126

[flagged]

This is the correct assessment. This is not up to the open source community or individual projects to "figure out", any more than its up to me to figure out how not to get spam email.

Yeah well, our corporate overlords have decided that you're going to take your slop whether you want it or not, so its very much up to us to figure out. Capitalism isn't going to jump off the disaster train any time soon

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#154

Post by 20k »

btilly wrote: post #27
arecsu wrote: post #16

Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort…

ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat. Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating ot…

>what happens if a single AI manages to get through to contributor

Then they'll get removed by the humans? Its about cutting down work, not about eliminating the work entirely

The current approach removes about 99% of their overhead it would seem. If they have to do a few manual interventions here and there, that seems like a huge win overall

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#155

Post by rglullis »

corps_and_code wrote: post #150
rglullis wrote: post #141

Earlier quoted context omitted.

I believe you are asking me in jest, but if you are genuine, this is what I would add to my CONTRIBUTING.md ``` # FIrst-time contributors Due to the increased number of AI bots and low-effort contributions, we are being forced to add some friction for first-time contributors. PRs are closed for anyone not explicitly added to our list of authorized users. To be accepted in the list, you must do one of the following: -…

Oh no, I'm being genuine. Documenting the process itself wasn't what I was curious about, I was wondering how you'd go about your last bullet. Accepting lots of currencies can be hard, but I guess I'm not super familiar with online escrow services. I'm not sure how simple they can make that process, or who would pay the cost of using them (I assume they're not free). I was also wondering how automated or manual you w…

Yes, I'm assuming that adding requirement for payment would bring the number of requests down to a level which I could manage with a simple spreadsheet.

Paypal/SEPA transfers are free in Europe. And even if I lived in the US and had to pay a small processor fee, I'd be more than willing to cover the $0.50 in fees if that meant I was receiving contributions from people who went through all the trouble.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#156

Post by ildari »

IshKebab wrote: post #33

That's a neat way to interface with GitHub's authentication system, but I don't see how they've solved the fundamental problem because their whitelisting process is just "click ok fine 10 times". Why won't the slop peddlers just do that too?

click ok fine 10 times + captcha seems to be working fine

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#158

Post by LtWorf »

rglullis wrote: post #147
LtWorf wrote: post #144

Earlier quoted context omitted.

I think the intersection of the set of people able and interested in contributing and those who are willing to figure out cryptocurrencies is the empty set.

I quick visit to https://gitcoin.co/campaigns will show you that you are wrong. Hundreds of projects funded by even more people. Mind you: that's on one of the most convulated ways there is to get involved, because it involves a bunch of smart contract operations and on-chain voting. If we are talking about crypto only as a payment network, things are even simpler.

It's cheating if the projects using it are cryptocurrency related :)

A generic python library used by generic people who have no interest in this field is something else.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#160

Post by atomflunder3000 »

altairprime wrote: post #60
btilly wrote: post #27

Earlier quoted context omitted.

ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat. Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating ot…

ELO is a bad fit because it requires competition between submitters; but if the idea is interpreted as “contributor karma score” or similar (not everyone’s familiar with the mathematical nature of ELO), then the way to close the loophole is to only consider voting inputs from the human project owner. This project chose to have people lie to a webform rather than lie to a git interface about using AI, so I don’t expec…

I think they were saying Elo system as kind of a general ranking system idea instead of the actual algorithm.

You could probably use some kind of pairwise ranking algorithm (like anything based on the Bradley-Terry model) to rate human vs. AI contributions, but that would take a lot of manual effort. Google is using it to (supposedly) improve their searching algorithms. They give testers two different versions and ask them what's better.

Post reply on HN