Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: adamsreview – better multi-agent PR reviews for Claude Code

github.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#61

Post by azurewraith »

nkmnz wrote: post #10

Great project! I’ve build something similar, not very clean and polished, but focussed around deterministic orchestration of multiple agents via typescript, because a coordinating agent was notoriously bad at things such as fetching relevant tickets and other context. One thing I struggle with so far, though, are the actual instructions for the review themselves. They are either too vague, leading to superficial or o…

You've touched on the core issue... prompt based orchestration is non-idempotent. Same input, different orchestration paths every run. I've had better experience enforcing tool restrictions in discrete phases with deterministic code over orchestrating via LLM. Prompt level instructions get ignored, it's much harder to ignore a protocol-level tool restriction. It's simple and yet oddly VERY effective.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#62

Post by azurewraith »

esperent wrote: post #40
lmeyerov wrote: post #28

Earlier quoted context omitted.

Yes, being comprehensive, so early or blatant cheapo findings do not distract from other ones. That's important for base results. Splitting in both file and task is (currently) important. Additionally, we run in a loop until it stops finding things, and as part of that, do test amplification when it does find any. We regularly see 3-8 rounds yielding valid results. IMO half the value is customization to your repo, so…

How do you prevent it from increasing scope? That's the main issue I've found from running loops like this. Each loop has ~7 agents, say, looking through different lenses (security, UX, performance, etc.). Each one notes a few issues, each issue gets fixed, you do 5 to 8 loops, as you say. Each individual item that gets fixed looks minor but when you add it all up at the end you've increased PR size and scope signifi…

I've had similar experiences when I throw a bunch of agents at a problem... some things get flagged but a lot gets truncated in the summarization step. Per-phase constraints solve this naturally, and I think the problem is better suited to be solved serially. Have each specialized 'review' phase scoped to only read and annotate (even better with a code-owners style read scoping) with max iterations in deterministic code. The scope can't creep past the constraints you've set for it. Scope explosion comes from agents having unbounded tool access and no transition gates between phases... it will overreach if given the opportunity to

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#64

Post by adamthegoalie »

michaelsalim wrote: post #51
adamthegoalie wrote: post #39

Does anyone have an open PR on a public repo? I'll run this against your PR for you with my CC credits as a sort-of benchmark! Send me your PR link :) I'm going to create one on one of my other repos meanwhile and add a link to the review when it's ready.

Quite a huge one, but here's one :) https://github.com/Vija02/TheOpenPresenter/pull/170

Yesss! On it now! 1-2 hours you’ll get a comment

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#65

Post by adamthegoalie »

quantumadopter wrote: post #56

[flagged]

Well, it doesn’t give you an opinion on whether you should merge or not. It gives you a list of issues along with details about those issues, such as fix hints, and whether you need human attention before fixing.

It’s all given to you in a structured file, in your chat, and as a nicely formatted PR comment.

Still up to the human to decide whether and when to merge based on the output!

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#66

Post by adamthegoalie »

phonon wrote: post #48
adamthegoalie wrote: post #31

Earlier quoted context omitted.

That's a great idea. I had trouble finding anything like this, a benchmark made for (AI) code reviewers. I had expected to find something like an eval harness available on GitHub, but couldn't find it. Any suggestions? Or maybe we/I/someone should build something like this? I suppose one challenge is that if it's going to be publicly available, it would also be easy to cheat, but still seems it would be useful if peo…

https://www.codereviewbench.com/ https://codereview.withmartian.com/

I will be trying these and will report back!

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#67

Post by adamthegoalie »

michaelsalim wrote: post #51
adamthegoalie wrote: post #39

Does anyone have an open PR on a public repo? I'll run this against your PR for you with my CC credits as a sort-of benchmark! Send me your PR link :) I'm going to create one on one of my other repos meanwhile and add a link to the review when it's ready.

Quite a huge one, but here's one :) https://github.com/Vija02/TheOpenPresenter/pull/170

Your review is ready sir

https://github.com/Vija02/TheOpenPresenter/pull/170#issuecom...

This was claude-only, no codex / --ensemble mode. It ran against the branch's base, which is 1 behind main.

Would love your feedback when you have the chance!

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#68

Post by quantumadopter »

adamthegoalie wrote: post #65
quantumadopter wrote: post #56

[flagged]

Well, it doesn’t give you an opinion on whether you should merge or not. It gives you a list of issues along with details about those issues, such as fix hints, and whether you need human attention before fixing. It’s all given to you in a structured file, in your chat, and as a nicely formatted PR comment. Still up to the human to decide whether and when to merge based on the output!

[flagged]

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#69

Post by adamthegoalie »

dayjaby wrote: post #55
adamthegoalie wrote: post #32

Earlier quoted context omitted.

On small PRs (small features / changes ~hundreds to thousands of lines), I'd say around 500,000 total tokens. On large PRs (new feature sets for apps ~10,000-30,000 lines), around 2-3 million total tokens. By the way, I should have mentioned in my original post, adamsreview counts tokens used by sub-agents across the stages, and tells you at the end of each stage the total used so far.

Since when are thousands of lines of code changes "a small PR"?

Fair point. But since Claude Code I guess? :D
Post reply on HN