Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Reviews Don't Scale

medium.com

Post reply on HN
Print view View original

11–15 of 15 posts

Re: Code Reviews Don't Scale

#11

Post by g-b-r »

surprisefox wrote: post #8
dieselgate wrote: post #6

Earlier quoted context omitted.

Right on, I've been quite bored from 8-4 hours lately and have been commenting a lot around forums etc and can relate. Writing articles like this is more productive. I agree about the AI comments, it's basically written in the HN Guidelines "Please don't complain about tangential annoyances" or multiple others like "Please don't post shallow dismissals, especially of other people's work. A good critical comment teach…

Very relatable. I've definitely found myself spending more time in discussions recently too. I kind of take the AI comment as a compliment to be honest. Its a shame the message can get dismissed so quickly because of it but I understand why people are suspicious now

Well, brand new account, first blog entry, and at least a not but, had I not said that someone else would have ;)

I wasn't sure, but I've been burned by commenting slop before, so I stressed that I knew I might be debating with an AI ;)

Re: Code Reviews Don't Scale

#13

Post by surprisefox »

g-b-r wrote: post #11
surprisefox wrote: post #8

Earlier quoted context omitted.

Very relatable. I've definitely found myself spending more time in discussions recently too. I kind of take the AI comment as a compliment to be honest. Its a shame the message can get dismissed so quickly because of it but I understand why people are suspicious now

Well, brand new account, first blog entry, and at least a not but, had I not said that someone else would have ;) I wasn't sure, but I've been burned by commenting slop before, so I stressed that I knew I might be debating with an AI ;)

I get it :) appreciate you being open minded

Re: Code Reviews Don't Scale

#14

Post by othmanosx »

I think AI should complement the human review, not replace it, and I think everyone already agrees with me on this.

The problem with AI reviews, is that they add a lot more text to read, the PR description is another wall of text, test steps are very verbose, most comments are just noise.

I believe AI should instead help pointing the dev to the important stuff and help identify noise, this would actually make the review go faster.

since the introduction of AI, the bottleneck for shipping code shifted from producing code to reviewing it, and the PR reviewing experience on Github is lackluster, leaving a lot to be desired.

my team now ships too much code and it sits in review for too long, reviewing AI code is not easy and we need a better tool to review code, especially on large PRs, Github no longer meets my needs or use cases so I had to build something myself.

I got frustrated enough that I built my own reviewer around this idea: instead of adding commentary, it groups files into folders by complexity, surfaces the handful of changes that actually need deep review, and marks the mechanical noise so you can blow through it. On migration PRs it cut my review time down a lot.

As a web dev, I know what works and what doesn't, I hated how AI bloats the PR with more text, so I changed that to make the AI help me categorize the files into folders, splitting them by complexity, and pointing me to the important stuff I need to review indepth, and flag the noise so I can skim it faster, that sped up reviewing time immensely, especially on migration PRs where most is mechanical name changes.

I can share more if people are curious, I'd rather not turn this into an ad.

Re: Code Reviews Don't Scale

#15

Post by TomOwens »

Having a bottleneck with AI-generated code is probably a good thing.

Consider before AI. In cases where code was written by a trusted human, you had confidence that they were doing the right stuff - the people understood the context and requirements, the changes were based on those requirements, and tests were updated. You could rely on some basic automated checks to run a linter and static analyzer, execute the tests, and check test coverage. You can defer longer-running tests to a later stage, but people can merge their work in and collaborate. You can have blocking reviews for untrusted outside contributors or new employees and review the rest later.

After AI, the input is a mess. I haven't personally seen the worst of it, but I've read about people generating code before any requirements development or analysis. Code is generated without understanding what it is supposed to do, which also prevents good testing. Even with good requirements, people generate solutions that aren't right, and the generation step may remove failing tests or make unintended changes that aren't covered by tests. So much code is generated from untrusted sources and rubber-stamped. Introducing a bottleneck where a human can slow the process down is a good thing here.

Maybe if we can get people to understand how to use AI tools responsibly and have humans take full accountability for the output and changes, we can get back to the old way. Trusted humans and an AI agent assistant can make changes that are solid enough to merge. Independent control over the pipeline and basic automated checks would give just enough confidence in the change, and delayed review before deploying or enabling the feature would work. But not until trusted humans actually maintain oversight over their tools.

Post reply on HN