Why AI will never replace human code review
graphite.dev
Why AI will never replace human code review
1–10 of 23 posts
Re: Why AI will never replace human code review
#2Also, humans will not replace AI code review, because humans get tired and lazy, and can't be scaled up to cover the need.
Re: Why AI will never replace human code review
#3I think an AI can help with a code review. I see tones for submitted CR requests for stuff that flat out doesn't compile or pass tests.
Re: Why AI will never replace human code review
#4I think an AI can help with a code review. I see tones for submitted CR requests for stuff that flat out doesn't compile or pass tests.
is that AI or just actually running CI?
Re: Why AI will never replace human code review
#5Rory Sutherland calls this the "doorman fallacy".
We look at a doorman and might naively think their function is to open doors so if we can replace their function, we can replace the person. Then we build an automation, fire the doorman and subsequently discover the doorman was responsible for a multitude of social tasks, like taking in the mail, co-ordinating services, providing small tasks for the residents etc. and that physically opening and closing doors was actually the least important part of their job.
Similarly, we think the purpose of code review must obviously be for reviewing code until we look deeper and understand the sociological purposes of code review.
Technologists have a bad habit of entering into a field they don't know, observing but not talking to any of the people they're trying to automate and assuming only the most legible parts of anyone's job are important to grok. It's important to understand that the value any job or function brings can often be totally opaque to an outsider and it requires actually talking to people and understanding total value chains to fully understand where technology can be used to improve things.
Re: Why AI will never replace human code review
#6Also, humans will not replace AI code review, because humans get tired and lazy, and can't be scaled up to cover the need.
If humans can't scale to review, how are they scaling to code?
Humans how code should review, or the whole point of code review is pointless, and everyone should just push to HEAD and revert on failure
Re: Why AI will never replace human code review
#7I think an AI can help with a code review. I see tones for submitted CR requests for stuff that flat out doesn't compile or pass tests.
is that AI or just actually running CI?
could use both though, failing CI->bot->simple summary of why CI failed. even better if this happens before you've requested a reviewer
Re: Why AI will never replace human code review
#8This well-thought article restricts the notion of code review to pull request assessment - diff file - on an existing code base, perhaps a mostly human-written legacy code base.
As a Java code generation researcher, I would appreciate any constructive comments regarding the automated code review of automated changes made to an entirely generated code base, whose live specifications and requirements have been modified in order to generate the code being reviewed.
Re: Why AI will never replace human code review
#9I think an AI can help with a code review. I see tones for submitted CR requests for stuff that flat out doesn't compile or pass tests.
is that AI or just actually running CI?
I really like the general take that LLMs scanning PRs is simply "zero config CI." We already have a great paradigm for this; we don't need to reinvent a new category. In that light, we can weigh its value more as a fuzzy linter, rather than a be-all-end-all.
Re: Why AI will never replace human code review
#10This well-thought article restricts the notion of code review to pull request assessment - diff file - on an existing code base, perhaps a mostly human-written legacy code base. As a Java code generation researcher, I would appreciate any constructive comments regarding the automated code review of automated changes made to an entirely generated code base, whose live specifications and requirements have been modified…
Fascinating - are you imagining a sort of adversarial AI situation, where one LLM bot writes the PRs, and a different one reviews them, leading to an organically improving codebase? Kind of a cool idea.