Live data from Hacker News

Ask HN: AI-generated spam pull requests?

news.ycombinator.com

1–10 of 27 posts

Ask HN: AI-generated spam pull requests?

#1
Are you seeing pull requests in open source projects that are clearly AI-generated?

We recently had someone open a pull request to our open source project and the code and the explanation of the code was clearly AI generated. It was obvious that the code doesn't work and the person had not tested the code. We do not know what the end goal of the person was but we confronted the person and closed the pull requests.

Has any other open source projects experienced this? What did you do?

Re: Ask HN: AI-generated spam pull requests?

#4
> We do not know what the end goal of the person was but we confronted the person and closed the pull requests.

Maybe this was a naïve attempt at inflating their GitHub numbers? Some people use those as a credibility measure when applying to jobs or getting clients.

Re: Ask HN: AI-generated spam pull requests?

#5
If the AI PR covered the bases within one standard deviation of your other (human) PRs, would you care?

Formerly "hand wavy" questions about humanity, cognition, awareness are now showing up right in front of us. They are transforming into things like (a) is this PR worth my time? (b) does it introduce legal / license risk? (c) what principles were considered during its creation and so on.

Re: Ask HN: AI-generated spam pull requests?

#6

> We do not know what the end goal of the person was but we confronted the person and closed the pull requests. Maybe this was a naïve attempt at inflating their GitHub numbers? Some people use those as a credibility measure when applying to jobs or getting clients.

Hacktoberfest should be interesting this year

Re: Ask HN: AI-generated spam pull requests?

#8
post #5

If the AI PR covered the bases within one standard deviation of your other (human) PRs, would you care? Formerly "hand wavy" questions about humanity, cognition, awareness are now showing up right in front of us. They are transforming into things like (a) is this PR worth my time? (b) does it introduce legal / license risk? (c) what principles were considered during its creation and so on.

The dose makes the poison. Do you remember when digital ocean (or someone similar) did that promotion thing for opening PRs/contributing to open source projects? It was a nightmare for maintainers because they were inundated with outright spam and also well-intentioned but poorly executed PR attempts by hapless newbies.

I think one reason that open source even kind of works is because "people contributing to open source" has a shit-ton of positive selection bias "baked in". The type of person liable to open a PR to an open source project is probably several standard deviations (or, at least one, right?) above the average developer. So that probably has the general effect of making reviewing a random cold-open PR less onerous of a task.

But if we cross into a world where the average value of a PR opened with a project drops substantially - either due to AI or due to a permanent advertising campaign from some company rewarding badges for opening open-source PRs - I wouldn't be surprised to see lots of open projects close/ignore github PRs and start doing something that looks more like how Linux handles it, where it's a lot more social-based and puts some of that positive-selection-bias filter back in place.

Re: Ask HN: AI-generated spam pull requests?

#9
post #5

If the AI PR covered the bases within one standard deviation of your other (human) PRs, would you care? Formerly "hand wavy" questions about humanity, cognition, awareness are now showing up right in front of us. They are transforming into things like (a) is this PR worth my time? (b) does it introduce legal / license risk? (c) what principles were considered during its creation and so on.

The dose makes the poison. Do you remember when digital ocean (or someone similar) did that promotion thing for opening PRs/contributing to open source projects? It was a nightmare for maintainers because they were inundated with outright spam and also well-intentioned but poorly executed PR attempts by hapless newbies. I think one reason that open source even kind of works is because "people contributing to open sou…

Indeed, when I saw this, I thought "Hacktoberfest is going to have problems this year."

Re: Ask HN: AI-generated spam pull requests?

#10

Relatedly, the Rails codebase recently received a (clearly marked) AI-generated pull request: https://github.com/rails/rails/pull/47708 At a glance, it looks like it's been mostly well-received and has not yet been immediately closed as spam.

Oh... I already see this commit being used by MS in the Copilot lawsuit.
Post reply on HN