Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We stopped AI bot spam in our GitHub repo using Git's –author flag

archestra.ai

Post reply on HN
Print view View original

191–200 of 254 posts

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#191

Post by karussell »

TuxSH wrote: post #177
drusepth wrote: post #169

Earlier quoted context omitted.

What is the benefit of deleting a PR over just closing it? It seems like closing has the benefit of signaling what kinds of PRs aren't acceptable, which deleting would lose.

Closing a PR or issue still makes it discoverable in PR/issue search results, as opposed to deleting an issue.

This. But OP wanted special requirements to open a PR. I.e. if those requirements are not met the PR is never visible to all and so admins can reject spam PRs without giving them a platform.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#192

Post by rglullis »

godelski wrote: post #184
rglullis wrote: post #149

Earlier quoted context omitted.

> Doesn't that put us into the same position? Of course not, because the number of low-quality PRs with $10 attached to it will be lower than whatever number of PRs are being created now.

You also lose out on a lot of would-be PRs. By people who don't have the money, don't have trust, or have a visceral "fuck you" stance. There's a lot more reasons that this suggestion creates a gate that dissuades the people you want. I stand by that the solution is naïve, but you're welcome to give it a try on your projects. I'm sure it'll be effective at reducing a lot of spammers, but I'm also pretty convinced it'…

> You also lose out on a lot of would-be PRs.

I am more concerned about the sustainability of the projects as a whole than trying to optimize the number of potential random PRs.

> you're welcome to give it a try on your projects.

It's not exactly the same, but in a way I'm already doing that with Communick. I'm running one of the few Matrix and Fediverse services where members must pay to have access. Up until last week, I was giving 14 days as a free trial period and no deposit/confirmed subcription required. But now because of AI bots, I dropped it and I am collecting payment info before activating any account.

If I were playing the startup playbook, that would be insane. It's already crazy to try to charge something that people are used to get for free; my conversion rates are already low, requiring credit card info will make them even lower.

In the end of the day, I don't care. At first I was really hoping this would be something profitable, right now I just keep it running because I can. Even with the small number of users Ithe servers get, I get enough to cover the better part of operational costs, I get to sharpen my devops skills and have a test lab to learn what is that people really want (i.e, they are willing to pay to have solved) vs what people claim to be a problem.

All in all, the Communick instances are not going to win any popularity contest, but my servers have been up and running without major issues or drama for more than 6 years, and that's a lot more than I can say for all the other servers that have come and gone because the admins tried to play the numbers game.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#193

Post by rglullis »

Lyrkan wrote: post #183
rglullis wrote: post #176

Earlier quoted context omitted.

> A PR is me giving my time to the project Unfortunately, the issue is that time is not enough of a filter anymore. The time from machines is basically worthless compared to yours, so you need to give something else, and that something else needs to be something that shows you have actual skin in the game.

> so you need to give something else Well no, they don't need to. As they said they could just do something else instead of contributing (and I know I would too). Your proposal would just end up killing those open source projects even more than what you are trying to solve.

> Your proposal would just end up killing those open source projects

Name 3 open source projects that are dependent on a continuous stream of first-time contributors to keep going.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#194

Post by rglullis »

LtWorf wrote: post #179
rglullis wrote: post #161

Earlier quoted context omitted.

Coinbase reports more than 100+ user accounts worldwide. Kraken has ~10M. Also, we are talking about people who are tech-savvy enough to be interested in participating in a FOSS project. Opening an account at an exchange is not rocket science.

No, getting the money out of an exchange if you are in a country USA doesn't like is rocket science. Which was the whole point of using crypto rather than money.

The exchanges are for the people that can send the money, not receive it.

And those who are living in countries the USA doesn't like will probably have no issue to learn how to work with crypto. Of all the complex things they need to do to work around the restrictions, setting up a wallet barely registers.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#195

Post by rglullis »

backwardsponcho wrote: post #185
rglullis wrote: post #163

Earlier quoted context omitted.

Oh, give me a break. No one is taking the ability from others to fork the repo. If these exceptional cases really were to happen, how fast would it be for someone else to notice and do one of (a) notify the maintainers to get this particular user whitelisted or (b) front the entry costs?

Sounds like bandaids on top of bandaids, at which point you start to wonder if the idea is fundamentally broken.

> Sounds like bandaids on top of bandaids

And a registration system that amounts to a more complicated captcha doesn't? How long until someone starts farming accounts and run bots that jump through these hoops as well?

> wonder if the idea is fundamentally broken.

It's only "fundamentally broken" if you need to build a perfect system that needs to accept 100% of legit PRs without raising any level of friction.

But we don't need that. Pfand systems are not meant to be perfect, and they are not meant to the single solution to any problem involving the commons. They will not get rid of all bad behavior, but they will certainly bring it a global-scale problem down to levels that can be then managed by other smaller, context-aware systems.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#196

Post by lmm »

catlifeonmars wrote: post #100
stronglikedan wrote: post #67

Earlier quoted context omitted.

From what I've seen in the comments, it's definitely ELO, if not through ubiquity alone. Happens to the best of 'em!

Elo is nicer as it gives a nod to the inventor, no?

Some say the greatest compliment a mathematician can be paid is when a concept that bears their name is so ubiquitous that we no longer capitalise it. (E.g. Abel with the abelian group).

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#197

Post by godelski »

rglullis wrote: post #192
godelski wrote: post #184

Earlier quoted context omitted.

You also lose out on a lot of would-be PRs. By people who don't have the money, don't have trust, or have a visceral "fuck you" stance. There's a lot more reasons that this suggestion creates a gate that dissuades the people you want. I stand by that the solution is naïve, but you're welcome to give it a try on your projects. I'm sure it'll be effective at reducing a lot of spammers, but I'm also pretty convinced it'…

> You also lose out on a lot of would-be PRs. I am more concerned about the sustainability of the projects as a whole than trying to optimize the number of potential random PRs. > you're welcome to give it a try on your projects. It's not exactly the same, but in a way I'm already doing that with Communick. I'm running one of the few Matrix and Fediverse services where members must pay to have access. Up until last w…

  > trying to optimize the number of potential random PRs.
You're misrepresenting my comment. I didn't say at need to optimize, just consider. Don't strawman me here

You can't just hand wave them away as if this isn't an important factor. If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs. Problem solved! But that's not a real solution either

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#198

Post by marcus_holmes »

krupan wrote: post #78

This is what we get for telling everyone how amazing AI is at writing code. It started with the people selling AI and for some reason tons of independent developers, some quite well respected in our field, piled on. Facebook now laying people off and saying it's because AI is just so good adds more fuel to the fire. Now you have a bunch of people fully confident that their AI friend is pumping out amazing code and su…

No, this is a result of unintended consequences.

We made "Github contributions" a metric for people applying for dev jobs. So, of course, because devs are the kind of people we are, they started working out how to game that metric.

Some folks decided to start paying bounties on bug fixes, features, etc. Those bounties are fairly trivial by western standards, but are significant for developing countries. This creates a new career for developers; racing to collect the bounties on offer.

LLMs have exacerbated these problems by allowing existing people doing this to do it faster, and also allowing more people to pretend to be software developers and get in on the action.

If we stopped allowing LLM-authored contributions we'd still have too many shitty PRs. It would just be back to pre-LLM levels of "too many".

The answer is to make Github contributions valueless. Stop paying bounties, and stop using them to assess candidates.

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#199

Post by rglullis »

godelski wrote: post #197
rglullis wrote: post #192

Earlier quoted context omitted.

> You also lose out on a lot of would-be PRs. I am more concerned about the sustainability of the projects as a whole than trying to optimize the number of potential random PRs. > you're welcome to give it a try on your projects. It's not exactly the same, but in a way I'm already doing that with Communick. I'm running one of the few Matrix and Fediverse services where members must pay to have access. Up until last w…

> trying to optimize the number of potential random PRs. You're misrepresenting my comment. I didn't say at need to optimize, just consider. Don't strawman me here You can't just hand wave them away as if this isn't an important factor. If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs. Problem solved! But that's not a real solution either

> You can't just hand wave them away as if this isn't an important factor..

There are plenty of ways to indicate in the project that the Pfand is meant as one way to filter out bad actors, but it doesn't mean that it should be the only way to accept external contributions. You can find somewhere else on the thread where I listed some alternatives that can be used as well.

> If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs.

Yes, and what is the problem with this solution? That's what many projects are doing and many more will do. They will close access to non-members and only accept someone new when they have some type of social proof. [0] And that is totally fine.

[0]: https://news.ycombinator.com/item?id=43423063

Re: We stopped AI bot spam in our GitHub repo using Git's –author flag

#200

Post by sdsd »

pbhjpbhj wrote: post #174
microtonal wrote: post #28

Earlier quoted context omitted.

I feel like GitHub should have a system where you can give out tokens that are valid for e.g. 1 PR. If someone shows to engage in meaningful discussion and has a good idea to address an issue/feature, you initially give them one PR token. If the PR is of good quality, you can give them a few more, until they are contributors that can just create PRs as they like. A similar system would be nice for issues, though I'm…

You could use a "OTP" to provide this: give out tokens ("OTP"), anyone with that token can submit, keep a record of the user (eg github username, email address) and token, run a bot to delete submissions that don't have a valid token, check the token+user pair, if there is a mismatch blacklist the user that token was given to whilst removing the PR.

If only there were a simple way to make tokens that weren't fungible and could be given to others

/s

Post reply on HN