There is an AI code review bubble • Hacker News Board

Re: There is an AI code review bubble

#131

Post by daotoad » Tue, Jan 27, 2026, 1:21 AM UTC

zenolijo wrote: post #83

Quarrelsome wrote: post #75

Earlier quoted context omitted.

> but the signal to noise ratio is poor Nail on the head. Every time I've seen it applied, its awful at this. However this is the one thing I loathe in human reviews as well, where people are leaving twenty comments about naming and then the actual FUNCTIONAL issue is just inside all of that mess. A good code reviewer knows how to just drop all the things that irk them and hyperfocus on what matters, if there's a fun…

Naming comments can be very useful in code that gets read by a lot of people. It can make the process of understanding the code much quicker. On the other hand, if it's less important code or the renaming is not clearly an improvement it can be quite useless. But I've met some developers who has the opinion of reviews as pointless and just say "this works, just approve it already" which can be very frustrating when i…

Naming comments are useful when someone catches something like:

1. you are violating a previously agreed upon standard for naming things

2. inconsistent naming, eg some places you use "catalog ID" and other places you use "item ID" (using separate words and spaces here because case is irrelevant).

3. the name you chose makes it easy to conflate two or more concepts in your system

4. the name you chose calls into question whether you correctly understood the problem domain you are addressing

I'm sure there are other good naming comments, but this is a reasonable representation of the kinds of things a good comment will address.

However, most naming comments are just bike shedding.

Re: There is an AI code review bubble

#132

Post by lifetimerubyist » Tue, Jan 27, 2026, 1:30 AM UTC

Haven’t used a single one that was any good. Basically a 50/50 crapshoot if what they are saying makes any sense at all, let alone it being considered “good” comments. Basically no different than random chance.

Re: There is an AI code review bubble

#133

Post by ex-aws-dude » Tue, Jan 27, 2026, 1:41 AM UTC

I find a lot of times with co-pilot it calls out issues where if the AI had more context of the whole codebase it would realize that scenario can’t actually occur.

Or it won’t understand some invariant that you know but is not explicit anywhere

Re: There is an AI code review bubble

#134

Post by shagie » Tue, Jan 27, 2026, 1:45 AM UTC

tayo42 wrote: post #121

shagie wrote: post #99

Earlier quoted context omitted.

The code works perfectly - there is no issue that a unit test could catch... unless you are spying on internally created objects to a method and verifying that certain functions are called some number of times for given data.

Sure and you can do that

Trying to write the easiest code that I could test... I don't think I can without writing an excessively brittle test that would break at the slightest implementation change.

So you've got this Java:

    public List someCall() {
        return IntStream.range(1,10).boxed().toList();
    }

    public List filterEvens(List ints) {
        return ints.stream()
                .filter(i -> i % 2 == 0)
                .toList();
    }

    int aMethod() {
        List data = someCall();
        return filterEvens(data.stream().filter(i -> i % 2 == 0).toList()).size();
    }

And I can mock the class and return a spied'ed List. But now I've got to have that spied List return a spied stream that checks to see if .filter(i -> i % 2 == 0) was called. But then someone comes and writes it later as .filter(i -> i % 2 != 1) and the test breaks. Or someone adds another call to sort them first, and the test breaks.

To that end, I'd be very curious to see the test code that verifies that when aMethod() is called that the List returned by SomeCall is not filtered twice.

What's more, it's not a useful test - "not filtered twice" isn't something that is observable. It's an implementation detail that could change with a refactoring.

Writing a test that verifies that filterEvens returns a list that only contains even numbers? That's a useful test.

Writing a test that verifies that aMethod returns back the size of the even numbers that someCall produced? That's a useful test.

Writing a test that tries to enforce a particular implementation between the {} of aMethod? That's not useful and incredibly brittle (assuming that it can be written).

Re: There is an AI code review bubble

#135

Post by nickitolas » Tue, Jan 27, 2026, 1:46 AM UTC

> In addition, success is generally pretty well-defined. Everyone wants correct, performant, bug-free, secure code.

I feel like these are often not well defined? "Its not a bug it's a feature", "premature optimization is the root of all evil", etc

In different contexts, "performant enough" means different things. Similarly, many times I've seen different teams within a company have differing opinions on "correctness"

Re: There is an AI code review bubble

#136

Post by 0xbadcafebee » Tue, Jan 27, 2026, 1:49 AM UTC

Hot take: Code review is an anti-pattern.

We spend a ton of time looking at the code and blocking merges, and the end result is still full of bugs. AI code review only provides a minor improvement. The only reason we do code review at all is humans don't trust that the code works. Know another way to tell if code works? Running it. If our code is so utterly inconceivable that we can't make tests that can accurately assess if the code works, then either our code design is too complicated, or our tests suck.

OTOH, if the reason you're doing code review is to ensure the code "is beautiful" or "is maintainable", again, this is a human concern; the AI doesn't care. In fact, it's becoming apparent that it's easier to replace entire sections of code with new AI generated code than to edit it.

Re: There is an AI code review bubble

#137

Post by SchemaLoad » Tue, Jan 27, 2026, 1:54 AM UTC

zenolijo wrote: post #83

Quarrelsome wrote: post #75

Earlier quoted context omitted.

> but the signal to noise ratio is poor Nail on the head. Every time I've seen it applied, its awful at this. However this is the one thing I loathe in human reviews as well, where people are leaving twenty comments about naming and then the actual FUNCTIONAL issue is just inside all of that mess. A good code reviewer knows how to just drop all the things that irk them and hyperfocus on what matters, if there's a fun…

Naming comments can be very useful in code that gets read by a lot of people. It can make the process of understanding the code much quicker. On the other hand, if it's less important code or the renaming is not clearly an improvement it can be quite useless. But I've met some developers who has the opinion of reviews as pointless and just say "this works, just approve it already" which can be very frustrating when i…

A lot of these comments are not pointing out actual issues, just "That's not how I would have done it" type comments.

Re: There is an AI code review bubble

#138

Post by SchemaLoad » Tue, Jan 27, 2026, 1:56 AM UTC

xmprt wrote: post #98

Quarrelsome wrote: post #75

Earlier quoted context omitted.

> but the signal to noise ratio is poor Nail on the head. Every time I've seen it applied, its awful at this. However this is the one thing I loathe in human reviews as well, where people are leaving twenty comments about naming and then the actual FUNCTIONAL issue is just inside all of that mess. A good code reviewer knows how to just drop all the things that irk them and hyperfocus on what matters, if there's a fun…

Human comments tend to be short and sweet like "nit: rename creatorOfWidgets to widgetFactory". Whereas AI code review comments are long winded not as precise. So even if there are 20 humans comments, I can easily see which are important and which aren't.

My coworker is so far on this spectrum it's a problem. He writes sentences with half the words missing making it actually difficult to understand what he is trying to suggest.

All of the non critical words in english aren't useless bloat, they remove ambiguity and act as a kind of error correction if something is wrong.

Re: There is an AI code review bubble

#139

Post by Arainach » Tue, Jan 27, 2026, 2:26 AM UTC

daotoad wrote: post #131

zenolijo wrote: post #83

Earlier quoted context omitted.

Naming comments can be very useful in code that gets read by a lot of people. It can make the process of understanding the code much quicker. On the other hand, if it's less important code or the renaming is not clearly an improvement it can be quite useless. But I've met some developers who has the opinion of reviews as pointless and just say "this works, just approve it already" which can be very frustrating when i…

Naming comments are useful when someone catches something like: 1. you are violating a previously agreed upon standard for naming things 2. inconsistent naming, eg some places you use "catalog ID" and other places you use "item ID" (using separate words and spaces here because case is irrelevant). 3. the name you chose makes it easy to conflate two or more concepts in your system 4. the name you chose calls into ques…

If the person reading the code doesn't quickly understand what's going on from the name or finds the name confusing, the name is poor and should be changed. It is way too easy for the author to be caught up in their mental model and to be unaware of their implicit assumptions and context and choose a name that doesn't make sense.

The bigger problem is people who feel ownership of shared codebases tied to their ego and who get angry when people suggest changes to names and other bits of interfaces instead of just making the suggested change.

If you get code review feedback, the default answer is "Done" unless you have a strong reason not to. If it's not obvious whether the name suggested by the author or the reader is better, the reader's choice should be taken every time.

Re: There is an AI code review bubble

#140

Post by rohansood15 » Tue, Jan 27, 2026, 2:35 AM UTC

dakshgupta wrote: post #4

personjerry wrote: post #2

I don't really understand how this differentiates against the competition. > Independence Any "agent" running against code review instead of code generation is "independent"? > Autonomy Most other code review tools can also be automated and integrated. > Loops You can also ping other code review tools for more reviews... I feel like this article actually works against you by presenting the problem and inadequately so…

> Independence It is, but when a model/harness/tools/system prompts are the same/similar in the generator and reviewer fail in similar ways. Question: Would you trust a Cursor review of Claude-written code more, less, or the same as a Cursor review of Cursor-written code? > Autonomy Plenty of tools have invested heavily in AI-assisted review - creating great UIs to help human reviewers understand and check diffs. Our…

> Would you trust a Cursor review of Claude-written code more, less, or the same as a Cursor review of Cursor-written code?

You're assuming models/prompts insist on a previous iteration of their work being right. They don't. Models try to follow instructions, so if you ask them to find issues, they will. 'Trust' is a human problem, not a model/harness problem.

> Our view is that code validation will be completely autonomous in the medium term.

If reviews are going to be autonomous, they'd be part of the coding agent. Nobody would see it as an independent activity, you mentioned above.

> Our first step towards making this easier is a native Claude Code plugin.

Claude can review code based on a specific set of instructions/context in an MD file. An additional plugin is unnecessary.

My view is that to operate in this space, you gotta build a coding agent or get acquired by one. The writing was on the wall a year ago.