Give Django your time and money, not your tokens

I don't think most maintainers are opposed to LLM-generated bug fixes or solutions _in general_, just the ones that are pure slop: Generated end-to-end by a Claude-maxed computer enthusiast who thinks that enough green boxes on their GitHub profile means they can somehow BS their way into a high-paying FAANG software engineer position. (Spoiler: it won't work.)

If I got a PR for one of my projects where the fix was LLM-generated, I wouldn't dismiss it out of hand, but I would want to see (somehow) that the submitter themselves understood both the problem and the solution. Along with all the other usual qualifiers (passes tests, follows existing coding style, diff doesn't touch more than it has to, etc). There's likely no one easy way to tell this, however.

This is a similar hot issue in academia right now. The ability to generate content in papers via llm is much easier than the ability to thoughtfully review them. There are now two tracks, at least in ICML that I saw, one for AI submitted papers and one for non AI submitted papers. And it works the same respectively for reviewers. However even for AI submitted papers, you cannot have only AI review it. Of course it needs human analysis, but still its tricky what you are going to get. And they are reviewing whether anonymity can still stand or if tying your credibilty to the review process is now necessary.

As for open source PRs, I wonder if for trust's sake you would need to self identify the use of AI in your response (All AI, some AI, no AI). And there would need to be some sort of AI detection algorithm flag your response as % AI. I wonder if this would force people to at least translate the LLM responses to their own words. It would for sure stop the issue of someone's WhatsApp 24/7 claw bot cranking out PR slop. Maybe this can lessen the reviewers burden. That being said, more thought is needed to distinguish helpful LLM use that enhances the objective vs unhelpful slop that places burden on the reviewer.

For instance I copy pasted the above to gemini and it produced an excellent condensing of my thoughts, "It is now 10x easier to generate a "plausible" paper or Pull Request (PR) than it is to verify its correctness."

I agree with the broarder thesis, although some people aren't buying more tokens for this and could just be using their existing plan's limits.

So people may be less likely to donate an extra amount beyond their "ai budget" to an OSS project for tokens. Large OSS projects are also likely to get free tokens from major providers anyway.

But I like the idea of crowdfunding specific features.

The best people I've worked with tended to go out of their way to make it as easy for me as possible to critique their ideas or implementations.

They spelled out exactly their assumptions, the gaps in their knowledge, what they have struggled with during implementation, behavior they observed but don't fully understand, etc.

Their default position was that their contribution was not worth considering unless they can sell it to the reviewer, by not assuming their change deserves to get merged because of their seniority or authority, but by making the other person understand how any why it works. Especially so if the reviewer was their junior.

When describing the architecture, they made an effort to communicate it so clearly that it became trivial for others to spot flaws, and attack their ideas. They not only provided you with ammunition to shoot down their ideas, they handed you a loaded gun, safety off, and showed you exactly where to point it.

If I see that level of humility and self-introspection in a PR, I'm not worried, regardless of whether or not an LLM was involved.

But then there's people that created PRs with changes where the stack didn't even boot / compile, because of trivial errors. They already did that before, and now they've got LLMs. Those are the contributions I'm very worried about.

So unlike people in other threads here, I don't agree at all with "If the code works, does it matter how it was produced and presented?". For me, the meta / out-of-band information about a contribution is a massive signal, today more than ever.

I wonder if checking for false statements or hallucinations is the first step to detect entirely LLM