Use AI for reviewing code especially when the diff is huge
simianwords.bearblog.dev
Use AI for reviewing code especially when the diff is huge
1–9 of 9 posts
Re: Use AI for reviewing code especially when the diff is huge
#2I can't tell if the title is a joke or not, the article seems to give some very good reasons why you shouldn't use an LLM to review pull requests
Re: Use AI for reviewing code especially when the diff is huge
#3If someone asks you for a code review and you turn your LLM loose to do it - you’re doing nothing. Just tell the requester you can’t be bothered and have them ask their LLM to do the review directly.
Relaying one LLMs output to another is very poor use of a human’s time.
Re: Use AI for reviewing code especially when the diff is huge
#4> I don't think you should waste time reviewing every single line of code in here and just use AI to review it!
> What you bring is the knowledge that the author nor the LLM doesn't know.
How can you possibly know what relevant context to provide the LLM unless you read the 10k loc? Now you've wasted double the time.
Re: Use AI for reviewing code especially when the diff is huge
#5If someone asks you for a code review and you turn your LLM loose to do it - you’re doing nothing. Just tell the requester you can’t be bothered and have them ask their LLM to do the review directly. Relaying one LLMs output to another is very poor use of a human’s time.
I don't think I was able to express my point well.
I'm specifically not asking you to run your LLM lose on it. I'm asking you to add the specific knowledge that you have that the author/LLM doesn't to the review. I tried to jot down such examples in the post.
Re: Use AI for reviewing code especially when the diff is huge
#6You can radically reduce your review burden but you can’t eliminate it - review is the part where you make your code good.
But whatever you see in review, you must add to a review checklist for the agent. You do this so you never have to check for that thing again.
Example: https://github.com/cadamsdotcom/CodeLeash/blob/main/.claude/...
(Use AGENTS.md or whatever agent docs you use, to tell your agent to review after it’s done working but before it stops.)
Congratulations! After you added hundreds more items to the agent’s review checklist, that work you did to build the checklist has automated the first 90% of the work. The second 90% is still.. reviewing the code. So no, you can’t escape code review.
It’s an asymptote. And it’s part of your job as a professional.
Re: Use AI for reviewing code especially when the diff is huge
#7> your line by line reviews have no place here.
I'll believe that once AIs stop outputting garbage code that spends 15 lines defensively checking for situations that can never occur or re-implementing their own URL-parsing logic instead of using a library.
Maybe Fable is better at this? Maybe there's a set of prompts or skills that will reduce these tendencies?
Re: Use AI for reviewing code especially when the diff is huge
#8I'm 100% opposed to AI generating my code, but I could see myself using it as an advanced linter. I suspect this will be considered best practices after the AI Slopocalypse
Re: Use AI for reviewing code especially when the diff is huge
#9If someone asks you for a code review and you turn your LLM loose to do it - you’re doing nothing. Just tell the requester you can’t be bothered and have them ask their LLM to do the review directly. Relaying one LLMs output to another is very poor use of a human’s time.
I don't think I was able to express my point well. I'm specifically not asking you to run your LLM lose on it. I'm asking you to add the specific knowledge that you have that the author/LLM doesn't to the review. I tried to jot down such examples in the post.
Your point was expressed well and makes sense. I tend to have the LLM explain the PR to me and I'll do a back and forth with it on the various decisions in the PR and their ramifications. That seems like a better way than reviewing every line, and agrees with your point in the post.