Stop sending me huge PRs; a rant
getsmall.xyz
Stop sending me huge PRs; a rant
1–10 of 116 posts
Re: Stop sending me huge PRs; a rant
#2Just an idea which I haven't personally tried: AI agents understand technical limitations, such as CI failures. Maybe make a CI job which checks that a PR has a reasonable size, and auto-reject with a polite message if it's not? Something like, "This PR size exceeds the limit of N lines that we accept for review; if you implement a big feature please consider splitting it in several smaller PRs." There are chances that it won't help, but it might!
Re: Stop sending me huge PRs; a rant
#3Just an idea which I haven't personally tried: AI agents understand technical limitations, such as CI failures. Maybe make a CI job which checks that a PR has a reasonable size, and auto-reject with a polite message if it's not? Something like, "This PR size exceeds the limit of N lines that we accept for review; if you implement a big feature please consider splitting it in several smaller PRs." There are chances th…
Yeah, that's a fun way to get massive stacks of PRs that are individually incomprehensible.
Re: Stop sending me huge PRs; a rant
#4A simple solution is to use a git hook that asks for confirmation if it is too big, with a suggestion to ask the user to have the agent split it up.
For OSS, my suggestion is to accept issues and specs do the implementation yourself. Warp.dev has a decent model of this in Github: https://github.com/warpdotdev/warp/blob/master/CONTRIBUTING....
Re: Stop sending me huge PRs; a rant
#5I know someone working on a smaller open source who has same thing. They have considered just blocking all PRs outside known contributors because AI spam even on their tiny open source project is too much.
At work, I've gotten into fights about PR approvals. If they are beyond us humans to review, screw it, remove the approver requirement and if CI passes, merge it.
Re: Stop sending me huge PRs; a rant
#6> why did you put it up for a human review at all then?
This seems to be the crux of the issue.
I'm guessing the most of the time, the answer is "because that's a mandatory gate to getting these changes into production". If the PR author doesn't see the value in review, it's going to be hard to convince them to write reviewable PRs.
If they're actually looking for human feedback, telling them how to submit PRs in a way that's amenable to human feedback is going to be a lot more successful.
Re: Stop sending me huge PRs; a rant
#7Just an idea which I haven't personally tried: AI agents understand technical limitations, such as CI failures. Maybe make a CI job which checks that a PR has a reasonable size, and auto-reject with a polite message if it's not? Something like, "This PR size exceeds the limit of N lines that we accept for review; if you implement a big feature please consider splitting it in several smaller PRs." There are chances th…
That message could also be generated by a PR line count rule and string replacement without the ai overhead.
Edit - apologies I misunderstood which side the ai agent should be on.
Re: Stop sending me huge PRs; a rant
#8In my experience the models perform substantially worse if asked to create small PRs or commits. They lack the ability to sequence work and understand dependencies efficiently enough to manage it – it's not that they can't do small PRs, it's that doing them takes vastly more resources which then hits context limits etc. And if you want to then go back and edit a stack of commits or PRs, rebasing work into the middle, that's even more. I don't think any of this scales linearly in the amount of code or number of commits.
This is all in addition to the fact that the models are generally poor at storytelling, because that requires a theory of mind of the person you're communicating with. Authoring for review is storytelling, it's making changes in such a way as to build confidence in the reviewer. I believe current LLMs are still years away from this.
In my opinion, if you can't do these things, you're just cosplaying software engineering. Vibe coding has its uses, as does LLM programming, I do a lot of this! But we're kidding ourselves and dropping our standards dangerously low if we think that this is software engineering.
Re: Stop sending me huge PRs; a rant
#9Just an idea which I haven't personally tried: AI agents understand technical limitations, such as CI failures. Maybe make a CI job which checks that a PR has a reasonable size, and auto-reject with a polite message if it's not? Something like, "This PR size exceeds the limit of N lines that we accept for review; if you implement a big feature please consider splitting it in several smaller PRs." There are chances th…
Yeah, that's a fun way to get massive stacks of PRs that are individually incomprehensible.
How? If one PR builds off another, won’t either:
PR 1 is size 400
PR 2 is size 400 + 400 new
PR 3 is size 800 + 400 new
At least that’s how it’s worked on teams I’ve worked on that have soft size limits.
Re: Stop sending me huge PRs; a rant
#10A simple solution is to use a git hook that asks for confirmation if it is too big, with a suggestion to ask the user to have the agent split it up. For OSS, my suggestion is to accept issues and specs do the implementation yourself. Warp.dev has a decent model of this in Github: https://github.com/warpdotdev/warp/blob/master/CONTRIBUTING....
Yes could be a pipe failure. Commonly used for coverage or security concerns, it could be also used for PR size.