Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A standard protocol to handle and discard low-effort, AI-Generated pull requests

406.fail

Post reply on HN
Print view View original

101–110 of 129 posts

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#101

Post by strogonoff »

zozbot234 wrote: post #80
strogonoff wrote: post #78

Earlier quoted context omitted.

Here’s what you could do if you somehow found yourself with an LLM-generated change to a codebase implementing a feature you want, and you wanted to do the most do expedite the implementation of that feature without disrespecting and alienating maintainers: 1. Go through all changes, understand what changed and how it solves the problem. 2. Armed with that understanding, write (by hand) a high-level summary of what c…

> Go through all changes, understand what changed and how it solves the problem. GP has said that they can't do this, since they're unfamiliar with the language and that specific part of the codebase. Their best bet AIUI is (1) ask the AI agent to reverse engineer the diff into a high-level plan that they are qualified to evaluate and revise, if feasible, so that they can take ownership of it and make it part of the…

Not being familiar with a part of a codebase is not an incurable condition.

If the conclusion you made of that there is no workaround, then let that be the entire point. The alternatives are to get over yourself and ask people to implement a feature, or to understand how to help and then help.

The former is what OP did; the latter I described what I see as an efficient way of achieving while making use of an LLM-produced PR.

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#102

Post by 0cf8612b2e1e »

msylvest wrote: post #69
0cf8612b2e1e wrote: post #6

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted exactly as how much we do not want to review your generated submission. I know it is in jest, but I really hate that so many documents include “shall”. The interpretation of which has had official legal rulings going both ways. You MUST use less ambiguo…

Around 1990 I attended ISO/JTC1 meetings generating standards for data communication. I still recall my surprise over the heated arguments over these words between the UK and the US delegations. (I'm from Denmark). In particular 'shall' and 'should' meant different things in English and American languages. ISO's first standard, ISO 1, states that ISO Standards shall be written in English so we had to do that, US dele…

Not a lawyer, but I have heard of a few American legal cases where the judge had to decide the meaning of a “shall”, so it does not seem well settled from my vantage.

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#104

Post by gwbas1c »

demorro wrote: post #89

> Q: "Isn't it your job as an open-source maintainer/developer to foster a welcoming community?" The answer to this implies that the requirement to be welcoming only applies to humans, but even in this hostile and sarcastic document, it doesn't go far enough. Open source maintainers can be cruel, malicious, arbitrary, whatever they want. They own the project, there is no job requirements, you have no recourse. Suck i…

The bigger issue is that that kind of statement is highly manipulative, and indicates someone who is playing politics instead of focusing on results.

The better response is to call the bluff, something along the lines of: "Running an open-source project is quite time consuming. Please don't waste our time with emotional manipulation to get your way. Instead, take the time to understand why your LLM-generated pull request is not useful. You can start by understanding that we have access to LLMs too, and realize that a significant amount of work needs to happen after an LLM proposes changes."

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#105

Post by gwbas1c »

reg_dunlop wrote: post #97

I'd love to hear some commentary about my idea surrounding this problem of AI PRs. Why not restrict the agents to writing tests only? If the tickets are written concisely, any feature request or fix could be reduced to necessary spec files. This way, any maintainer would be tasked with reviewing the spec files and writing the implementation. CI is pretty good at gatekeeping based on test suites passing...

That's not helpful, because:

1: LLMs can write awful tests

2: LLMs can write very useful code, especially when they are working in well-understood areas.

Which comes down to understanding that the LLM is a tool, and it's the job of the programmer to know how to use the LLM and evaluate its output.

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#106

Post by zoezoezoezoe »

youknownothing wrote: post #99

I think part of the deeper issue is that contributing to an OSS project has become a rite of passage, a way to strengthen your profile. If you need to have contributed to look good but you don't really care about the contribution itself then you resort to this kind of trick. We had a similar plague for vulnerability disclosures, with people reporting that they had "discovered" vulnerabilities like "if you call this f…

"I can do math really fast"

"okay, what's 137*243"

"132,498"

"not even close"

"but it was fast"

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#107

Post by JoshTriplett »

quotemstr wrote: post #92

Everyone is missing the obvious solution. Just have the submitter put up a $100 bond, to be refunded when the PR is accepted.

If there were any reasonable way to do something like this, I would love to see it.

Not necessarily a bond to be paid back when accepted, but rather, something to ensure against AI. "If you assert this is not AI, insert $10. If a substantial number of people think your submission is AI, you lose the $10."

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#109

Post by Muhammad523 »

yeswecatan wrote: post #93
mglvsky wrote: post #66

I prefer this policy: https://github.com/ghostty-org/ghostty/blob/main/AI_POLICY.m... > If you can't explain what your changes do and how they interact with the greater system without the aid of AI tools, do not contribute to this project. edit: added that quote

Good idea, though I'm not sure how to enforce it. You can ask an AI for that and then rewrite it in your own words.

Are LLM used (not users) even able to write in their own words?

Re: A standard protocol to handle and discard low-effort, AI-Generated pull requests

#110

Post by adampunk »

elcapitan wrote: post #77
y-curious wrote: post #9

Earlier quoted context omitted.

While I am with you on hoping, someone shamelessly PRing slop just is not going to feel shame when one of their efforts fail. It’s like being mean to a phone scammer, they just hang up and do it again

It's actually a valuable signal to the phone scammer if you're mean, because that means they can stop wasting their own effort of scamming you, and call somebody else.

That is hilarious. I love that you believe that. Being mean to a phone scammer is about your feelings and your time. They do not care. More importantly, the next person who calls you is not gonna be the same person. It’s like slamming the door on some Mormons expecting that that’ll be the end of that, when there’s just two entirely different Mormons that are gonna come by a month later. They cannot have a memory of the thing you did to the other Mormons.
Post reply on HN