Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop sending me huge PRs; a rant

getsmall.xyz

Post reply on HN
Print view View original

111–116 of 116 posts

Re: Stop sending me huge PRs; a rant

#111

Post by tartieret »

I have definitely noticed this in our organization as people use more AI tools. At first educating by providing feedback to the developers, then after a few repeat directly requesting that a PR is broken down if there is an obvious way to do it.

But the thing that really made the differences: 1) having a small github action that checks the size of a PR and leave a warning comment if it's large. Obviously some PR have to be large but then the developer has to justify 2) much better: getting early access to Github Stacked PR. we all like the experience and it solves a number of problems. without it even if you are disciplined and break down your work on several PRs, you end up having to deal with rebasing them one by one when the base moves. We modified Claude.md so that it tries to use it when it makes sense and now even AI generated changes result in stack of small PRs

I think it's worth investing in this as there are studies showing that review time is exponential with the size of PRs (or worse you are more likely to let defect go through on a large PRs). And AI agents are also better are reviewing smaller chunks.

I had personally several experiences of asking someone to break down a super large PR into smaller ones and found a defect in PR#2 which wasn't caught in the original AI driven PR review

Re: Stop sending me huge PRs; a rant

#112

Post by Anamon »

Tadpole9181 wrote: post #86
sipjca wrote: post #42

Earlier quoted context omitted.

As a maintainer just saying no and closing PRs is largely the solution

Sure, for FOSS. But at a business I'm not sure you'll stay employed long if you do this regularly and don't try to find ways to adapt to the new world we live in. To be clear, I don't have an answer either. Reviews have definitely become a bottleneck, and the agent-created code has absolutely not reached a "broadly trustworthy" state for complex problems and maintainable contributions.

Depends on the corporate environment. Refusing to look at oversized PRs makes good business sense, there's an argument to be made that working like this sabotages the quality of the product.

Re: Stop sending me huge PRs; a rant

#113

Post by Anamon »

throwaway2037 wrote: post #82
lubujackson wrote: post #79

Earlier quoted context omitted.

Like all extremists, he takes a mild view "if you have to explain the code, maybe it needs to be refactored" to "no comments ever". Never mind business logic or footguns that can't be fixed right now because of Reasons or "I tried to change this and it blew up because of some remote script calls it that isn't in the codebase" or any number of other very useful comments that inform engineers (and now LLMs) as guidepos…

You raise some good points here. When I write comments, I only write what is not obvious from code. Example: Imagine there is some bizarre business rule that some insurance contract is 6% cheaper to write on the last business day of the month. That is exactly the kind of thing I will write inline code comments about. Someone without deep knowledge of that code would be clueless about that specific business rule. I fo…

That's exactly how I see it. And 99% of LLM-generated comments are just that: annoyingly wordy repetitions of what's already in the code. Just redundant noise that is a waste of time to read. Usually because the LLM has no concept of the information that might actually be helpful in a comment.

The same goes for LLM PR descriptions and documentation. Those spaces were meant for additional information not already in the code, not superfluous prose.

Re: Stop sending me huge PRs; a rant

#114

Post by dlevine »

Arainach wrote: post #78
dlevine wrote: post #71

I think this is more a symptom of the problem than the actual problem. The issue is that we can generate tons of code using AI, but then are blocked on having humans review all of it. I don’t think we should auto-approve all of this code without human review - that clearly doesn’t work either. What I do think we need is probably at least two-fold 1) better ways to explain these big PRs to human reviewers. 2) better w…

> Things like auto generating walkthrough videos I'm trying to write this with respect, but please explain your thought process here. If your PR description is a video instead of a written explanation, I'm rejecting it without even reading the code.

It’s not an either/or. It’s doing both. The video makes it easier to understand what the code does. We have used Looms on PRs since before AI, and agents are beginning to be able to do this on PRs they generate.

Re: Stop sending me huge PRs; a rant

#115

Post by ArnaudDebray »

dlevine wrote: post #71

I think this is more a symptom of the problem than the actual problem. The issue is that we can generate tons of code using AI, but then are blocked on having humans review all of it. I don’t think we should auto-approve all of this code without human review - that clearly doesn’t work either. What I do think we need is probably at least two-fold 1) better ways to explain these big PRs to human reviewers. 2) better w…

Agree with you. Going even further, I'm wondering whether the Diff alone is still the right review artifact... On your 1), I see Entire.io is trying to build some useful building blocks (capturing all agent sessions and prompts to extract the human intent). With a friend, we're currently working on a tool to ease the review using entire's agent session

Re: Stop sending me huge PRs; a rant

#116

Post by oxedom »

forbidden404 wrote: post #110
oxedom wrote: post #90

I recently started stacking my PRs, makes life easier https://docs.github.com/en/pull-requests/how-tos/stacked-pul...

LLMs will still produce huge PRs even in a stack if you are not careful enough.

That's a different problem. The benefit of stacked PRs is that they let you review each piece on its own, which helps you/team keep mental alignment with changes in the codebase and disregard individual slop layers.
Post reply on HN