Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is an AI code review bubble

greptile.com

Post reply on HN
Print view View original

151–160 of 265 posts

Re: There is an AI code review bubble

#151

Post by Phlebsy »

eieio wrote: post #144
Quarrelsome wrote: post #75

Earlier quoted context omitted.

> but the signal to noise ratio is poor Nail on the head. Every time I've seen it applied, its awful at this. However this is the one thing I loathe in human reviews as well, where people are leaving twenty comments about naming and then the actual FUNCTIONAL issue is just inside all of that mess. A good code reviewer knows how to just drop all the things that irk them and hyperfocus on what matters, if there's a fun…

at my last job code review was done directly in your editor (with tooling to show you diffs as well). What this meant was that instead of leaving nitpicky comments, people would just change things that were nitpicky but clear improvements. They'd only leave comments (which blocked release) for stuff that was interesting enough to discuss. This was typically a big shock for new hires who were used to the "comment for…

> What this meant was that instead of leaving nitpicky comments, people would just change things that were nitpicky but clear improvements. They'd only leave comments (which blocked release) for stuff that was interesting enough to discuss.

This is my dream; have only had a team with little enough ego to actually achieve it once for an unfortunately short period of time. If it's something that there's a 99% chance the other person is going to say 'oh yeah, duh' or 'sure, whatever' then it's just wasting both of your time to not just do it.

That said, I've had people get upset over merging their changes for them after a LGTM approval when I also find letting it sit to be a meaningless waste of time.

Re: There is an AI code review bubble

#152

Post by colechristensen »

zmmmmm wrote: post #70

My experience with using AI tools for code review is that they do find critical bugs (from my retrospective analysis, maybe 80% of the time), but the signal to noise ratio is poor. It's really hard to get it not to tell you 20 highly speculative reasons why the code is problematic along with the one critical error. And in almost all cases, sufficient human attention would also have identified the critical bug - so hu…

One thing I've found to be successful is to

1) give it a number of things to list in order of severity

and

2) tell it to grade how serious of a problem it may be

The human reviewer can then look at the top ten list and what the LLM thinks about its own list for a very low overhead of thinking (i.e. if the LLM thinks its own ideas are dumb a human probably doesn't need to look into them too hard)

It also helps to explicitly call out types of issue (naming, security, performance, correctness, etc)

The human doesn't owe the LLM any amount of time considering, it's just an idea generating tool. Looking through a top ten list formatted as a table can be scanned in 10 seconds in a first pass.

Re: There is an AI code review bubble

#154

Post by dang »

bofadeez wrote: post #153

[flagged]

"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."

https://news.ycombinator.com/newsguidelines.html

Edit: Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

Re: There is an AI code review bubble

#155

Post by tayo42 »

shagie wrote: post #134
tayo42 wrote: post #121

Earlier quoted context omitted.

Sure and you can do that

Trying to write the easiest code that I could test... I don't think I can without writing an excessively brittle test that would break at the slightest implementation change. So you've got this Java: public List someCall() { return IntStream.range(1,10).boxed().toList(); } public List filterEvens(List ints) { return ints.stream() .filter(i -> i % 2 == 0) .toList(); } int aMethod() { List data = someCall(); return fil…

You mention the tools you can use to make it happen.

I think we're at the point where you need concrete examples to talk about whether it's worth it or not. If you have functions that can't be called twice, then you have no other option to test details in the implementation like that.

Yeah there's a tradeoff between torturing your code to make everything about it testable and enforce certain behavior or keeping it simpler.

I have worked in multiple code bases where every function call had asserts on how many times it was called and what the args were.

Re: There is an AI code review bubble

#156

Post by dcre »

rushingcreek wrote: post #60

Greptile is a great product and I hope you succeed. However, I disagree that independence is a competitive advantage. If it’s true that having a “firewall” between the coding agent and review agent leads to better code, I don’t see why a company like Cursor can’t create full independence between their coding and review products but still bundle them together for distribution. Furthermore, there might well be benefits…

It's a ridiculous argument. Complete gibberish. They're all just API calls. There's no incentive to be biased toward approving bad code.

Re: There is an AI code review bubble

#157

Post by tokioyoyo »

raincole wrote: post #78

I still think any business that is based on someone else's model is worthless. I know I'm sounding like the 'dropbox is just FTP' guy, but it really feels like that any good idea will just be copied by OpenAI and Anthropic. If AI code review is proven a good idea is there any reason to expect Codex or Claude Code to not implement some commands to do code review?

We do our review through Claude github actions. Works well.

Re: There is an AI code review bubble

#158

Post by tr_user »

Quarrelsome wrote: post #75
zmmmmm wrote: post #70

My experience with using AI tools for code review is that they do find critical bugs (from my retrospective analysis, maybe 80% of the time), but the signal to noise ratio is poor. It's really hard to get it not to tell you 20 highly speculative reasons why the code is problematic along with the one critical error. And in almost all cases, sufficient human attention would also have identified the critical bug - so hu…

> but the signal to noise ratio is poor Nail on the head. Every time I've seen it applied, its awful at this. However this is the one thing I loathe in human reviews as well, where people are leaving twenty comments about naming and then the actual FUNCTIONAL issue is just inside all of that mess. A good code reviewer knows how to just drop all the things that irk them and hyperfocus on what matters, if there's a fun…

Depends on what you're targeting

- If it's a rough PR, you're looking for feedback on direction rather than nitpicks.

- If it's in a polished state, it's good to nitpick assuming you have a style guide you're intending to adhere to.

Perhaps this can be provided in the system prompt?

Re: There is an AI code review bubble

#159

Post by m463 »

zmmmmm wrote: post #70

My experience with using AI tools for code review is that they do find critical bugs (from my retrospective analysis, maybe 80% of the time), but the signal to noise ratio is poor. It's really hard to get it not to tell you 20 highly speculative reasons why the code is problematic along with the one critical error. And in almost all cases, sufficient human attention would also have identified the critical bug - so hu…

> signal to noise ratio is poor

I think this is the problem with just about every tool that examines code.

I've had the same problem with runtime checkers, with static analysis tools, and now ai code reviews.

Might be the nature of the beast.

probably happens with human code reviews too. Lots of style false positives :)

Re: There is an AI code review bubble

#160

Post by las3r »

I would suggest you check out your Greptile discord and/or answer your messages on X where people are trying to reach you with problems and questions about your service. Unless that no longer matters.
Post reply on HN