Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Code Review – An AI-powered code review CLI tool

github.com

Post reply on HN
Print view View original

61–70 of 89 posts

Re: Open Code Review – An AI-powered code review CLI tool

#61

Post by embedding-shape »

pramodbiligiri wrote: post #60

A repo with the English translation of each of the rules files, using Google Translate: https://github.com/pramodbiligiri/open-code-review-rules . The original rules files (in Chinese): https://github.com/alibaba/open-code-review/tree/main/intern...

I guess since you copy-pasted your comment here, yet didn't include a more proper and correct translation (again), here is my other comment again:

And for comparison, here's a GitHub gist with three versions, first the original Chinese one, then the Google Translate version you put and finally a translated done with ChatGPT Pro: https://gist.github.com/embedding-shapes/7a51d565214bd676890...

Done that way mainly to see how the Google Translate version compared with a ChatGPT translation (revision: https://gist.github.com/embedding-shapes/7a51d565214bd676890...)

Re: Open Code Review – An AI-powered code review CLI tool

#62

Post by atestu »

eranation wrote: post #18
atestu wrote: post #3

We've been using Coderabbit, great deal ($30/mo/dev flat) and finds a lot. I also built a skill I call `/meta-review` that asks Codex, Cursor, and Gemini to review the code (I use Claude Code). It always finds little things claude & I missed. Coderabbit just came out with their own PR review UI that's great for big PRs, it groups files together etc. https://www.coderabbit.ai/blog/introducing-atlas-the-first-a...

Not sure why you got downvoted, and I have nothing against CodeRabbit, but this comment feels a bit like a paid ad :) How do you see CodeRabbit against other AI code review solutions? E.g. cubic.dev, Qodo, Graphite, Greptile, Baz, Augment Code... An alternative UI to GitHub is well overdue. But once someone will get it right, everyone will copy them...

That’s exactly why I’m getting downvoted, and I get it tbh. I knew people would ask for recommendations. It’s ok.

I haven’t used any of the tools you mentioned. We started using coderabbit just this year. The new PR review UI just came out. It’s made for big AI reviews which internally we’re trying to rein in. I like the direction they’re moving in with that, it uses AI to help you rather than bypass you. So you have the automated review that catches a bunch, and then they have a tool for you to step in and do your own review faster.

It’s funny there’s someone replying to me saying coderabbit is the best they’ve ever seen and in another thread someone else says it’s the worst. If that’s not AI for you… you just gotta try it

Re: Open Code Review – An AI-powered code review CLI tool

#65

Post by bobkb »

eranation wrote: post #29

Ran it on a subset of 10 of the 50 PRs in this benchmark https://codereview.withmartian.com - very good recall (~74%, e.g. found a lot of the golden issues) - not so good precision (~12%, e.g. lots of false positives) - the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)

False positives from the deterministic audits a very difficult problem to address. Comparing and deduplicating across different methods or LLM audits seems to the only way.

Re: Open Code Review – An AI-powered code review CLI tool

#66

Post by tirpen »

eranation wrote: post #29

Ran it on a subset of 10 of the 50 PRs in this benchmark https://codereview.withmartian.com - very good recall (~74%, e.g. found a lot of the golden issues) - not so good precision (~12%, e.g. lots of false positives) - the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)

Which LLM did you use? I assume that will make a pretty big difference.

Re: Open Code Review – An AI-powered code review CLI tool

#67

Post by lizhengfeng101 »

sfortis wrote: post #41

Is not working with gpt5.x models (Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.) which is hardcoded. I dont know why this is on the front page. My review-with-codex skill is working just fine, consuming my usage and not API tokens.

I'm sorry — we didn't expect to receive so much attention from the developer community so soon after open-sourcing the project. Some parts of the codebase are not yet fully polished. We are currently refactoring the LLM module and will address this as soon as possible. Once again, I sincerely apologize for the inconvenience.

Re: Open Code Review – An AI-powered code review CLI tool

#68

Post by eranation »

tirpen wrote: post #66
eranation wrote: post #29

Ran it on a subset of 10 of the 50 PRs in this benchmark https://codereview.withmartian.com - very good recall (~74%, e.g. found a lot of the golden issues) - not so good precision (~12%, e.g. lots of false positives) - the precision causes the F1 to tank (~20%, if this stays the same on the full 50 sample it would puts it almost last, even less than Kilo+Grok)

Which LLM did you use? I assume that will make a pretty big difference.

gpt-5-mini and gpt-5.5 (had to tweak the code a bit to make it work)

Surprisingly not as big of a difference as one would hope. It turns out that smarter models are more conservative. Smarter model / More thinking = slightly worse recall sometimes.

I think it says more about the benchmark itself perhaps. Reviews are highly opinionated. And it could be that the smarter models are actually better, just the “golden” state is very opinionated.

Re: Open Code Review – An AI-powered code review CLI tool

#69

Post by eranation »

lukaslalinsky wrote: post #26
atestu wrote: post #3

We've been using Coderabbit, great deal ($30/mo/dev flat) and finds a lot. I also built a skill I call `/meta-review` that asks Codex, Cursor, and Gemini to review the code (I use Claude Code). It always finds little things claude & I missed. Coderabbit just came out with their own PR review UI that's great for big PRs, it groups files together etc. https://www.coderabbit.ai/blog/introducing-atlas-the-first-a...

I've tried many AI code review tools. Nothing comes close to the depth of CodeRabbit reviews. It's the only such tool that can find real logical bugs. I'd love to be able to get Claude Code to do similar quality of review, but I can't get it right, no matter how I try.

Even if this was true, it’s hard to believe, and written a bit like an ad. Eg no vendor would get me to write a comment like this. All the more so, I tried CR and my team asked me to remove it. Maybe they got better but it’s a bit weird to me considering they only charge $20 and Claude Code say they estimate the same cost for a single PR for their competing product.
Post reply on HN