Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Code Review – An AI-powered code review CLI tool

github.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Open Code Review – An AI-powered code review CLI tool

#31

Post by krzyk »

eranation wrote: post #21
elpakal wrote: post #5

At a kill s@@s hackathon at work, I was able to build something that uses a node image installs claude code runs a /review-like command puts inline comments to PR deletes old comments when rerunning OCR seems cool, but overkill, and I'm definitely not using Code Rabbit after their CEO was on here acting snobbish a while back. Point being AI code review in Git** itself isn't hard to do and can add a lot of value quick…

Nothing against coderabbit or SaaS specifically, but this was one of the reasons I stopped using it https://kudelskisecurity.com/research/how-we-exploited-coder... It's very easy to build a basic code review tool. It's hard to build one that developers won't ask you to turn off because of false positives (or one that will miss your next escaped bug) I think if all the tool does is run a claude code level /review skil…

I think that in most cases you either agree on a PR comment or you don't. But it has to leave a mark in PR. This is how we do reviews, ignoring PR comment is one of the worst offenses one can make. I don't let it go.

Re: Open Code Review – An AI-powered code review CLI tool

#32

Post by akie »

eranation wrote: post #29

Ran it on a subset of 10 of the 50 PRs in this benchmark https://codereview.withmartian.com - very good recall (~74%, e.g. found a lot of the golden issues) - not so good precision (~12%, e.g. lots of false positives) - the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)

I would say that recall is the most important metric here though. I'd want it to catch all the issues.

False positives are easy to ignore.

Re: Open Code Review – An AI-powered code review CLI tool

#33

Post by krzyk »

faangguyindia wrote: post #4

If you've codex what does it add over codex's default app? I am confused. Can't you simply ask codex in another tab to just do a code review?

It can be used outside of local machine.

We built something similar, it looks for new PRs where the bot is added and does reviews. Makes the code more tuned toward similar rules. I can't assume that a developer run a code review tool himself (just as I don't assume he/she run a build - so we run builds also).

It is just another perspective for code review, besides human. Unfortunately it uses a lot of tokens, and considering that Anthropic, OpenAI and Github Copilot all moved to token based pricing, it is quite a money burner.

Re: Open Code Review – An AI-powered code review CLI tool

#34

Post by krzyk »

cheema33 wrote: post #14
faangguyindia wrote: post #4

If you've codex what does it add over codex's default app? I am confused. Can't you simply ask codex in another tab to just do a code review?

> Can't you simply ask codex in another tab to just do a code review? You are likely to get better results if you do not use the same model for review that wrote the code. I typically use Opus for code editing and GPT 5.5 for peer review using an automation with skills. Training set is different between models. If there are gaps in coverage in one model, you want a different model reviewing the work. The second model…

Results also depend on the prompt. You get different results if you ask to review the PR and focus on particular file than if you don't make it focus.

Or if you make it "be a security engineer" with particular focus points.

Or make it a grammar nazi, it will find way more typos than without such focus.

Of course all of those "focuses" needs to be in a separate context (agent/subagent) to make it work.

Re: Open Code Review – An AI-powered code review CLI tool

#36

Post by witx »

akie wrote: post #32
eranation wrote: post #29

Ran it on a subset of 10 of the 50 PRs in this benchmark https://codereview.withmartian.com - very good recall (~74%, e.g. found a lot of the golden issues) - not so good precision (~12%, e.g. lots of false positives) - the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)

I would say that recall is the most important metric here though. I'd want it to catch all the issues. False positives are easy to ignore.

What, no they're not. You still need to analyze them to understand they are false positives. It's time wasted

Re: Open Code Review – An AI-powered code review CLI tool

#37

Post by pi-victor »

i did something like this, but somewhat in reverse. you are the one that reviews the code and you instruct AI what to do through code review comments: https://parley.cloudflavor.io.

thinking about it, it would be funny to first run alibaba's tool and then run parley after.

posted it here a few days ago: https://news.ycombinator.com/item?id=48369782 i guess with AI there are too many Show HN now, and i never got any type of feedback.

Re: Open Code Review – An AI-powered code review CLI tool

#38

Post by viblo »

pi-victor wrote: post #37

i did something like this, but somewhat in reverse. you are the one that reviews the code and you instruct AI what to do through code review comments: https://parley.cloudflavor.io . thinking about it, it would be funny to first run alibaba's tool and then run parley after. posted it here a few days ago: https://news.ycombinator.com/item?id=48369782 i guess with AI there are too many Show HN now, and i never got any…

Just a small note, the font on your site is very annoying to read, the characters are not aligned horizontally (Windows w Chrome). Looks to be a scaling issue, if I zoom to 200% it shows fine.

Re: Open Code Review – An AI-powered code review CLI tool

#40

Post by onion2k »

witx wrote: post #36
akie wrote: post #32

Earlier quoted context omitted.

I would say that recall is the most important metric here though. I'd want it to catch all the issues. False positives are easy to ignore.

What, no they're not. You still need to analyze them to understand they are false positives. It's time wasted

Finding problems is optimizing for the customer. Avoiding false positives is optimizing for the developer. Which is right depends on your org's culture.
Post reply on HN