Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: adamsreview – better multi-agent PR reviews for Claude Code

github.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#31

Post by adamthegoalie »

parasmadan wrote: post #18

Why not just use an eval harness to prove this catches more real bugs? Benchmarks on actual bug classes would be far more convincing than comparing against /review.

That's a great idea. I had trouble finding anything like this, a benchmark made for (AI) code reviewers.

I had expected to find something like an eval harness available on GitHub, but couldn't find it.

Any suggestions? Or maybe we/I/someone should build something like this?

I suppose one challenge is that if it's going to be publicly available, it would also be easy to cheat, but still seems it would be useful if people agreed it's a good benchmark and could easily re-test tools themselves.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#32

Post by adamthegoalie »

thesimon wrote: post #8

> Runs against your regular Claude Code subscription (Max plan recommended) — unlike /ultrareview, which charges against your Extra Usage pool. How expensive is it to run in your experience? In $ or tokens?

On small PRs (small features / changes ~hundreds to thousands of lines), I'd say around 500,000 total tokens.

On large PRs (new feature sets for apps ~10,000-30,000 lines), around 2-3 million total tokens.

By the way, I should have mentioned in my original post, adamsreview counts tokens used by sub-agents across the stages, and tells you at the end of each stage the total used so far.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#33

Post by adamthegoalie »

moomin wrote: post #15

Is there a good way of adding in your own rules to the review? I’m always in the market for better review tools but I also need to check against internal coding stands and expectations,

Definitely!

adamsreview is mostly english language - since it is instructing your CC agents. While there are a number of python scripts and JSON storage, you - and your agent - would easily be able to add your own rules to this. It also respects your Claude.md and one of the lenses it already uses is checking for Claude.md compliance etc.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#34

Post by adamthegoalie »

Ozzie-D wrote: post #14

the irony of multi-agent code review is that the people who would use it are already the ones who care about code quality. the real problem is everyone else just hitting accept on whatever claude spits out without even reading the diff. tooling for review keeps getting better while the average review effort keeps going down.

Do you mean that, even though the tooling keeps getting better, people aren't putting the effort into using it?

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#35

Post by adamthegoalie »

stingraycharles wrote: post #16
docheinestages wrote: post #6

We seem to be fighting complexity with complexity. Does it really help?

This has been the gradual progression of the software world for the past half century, so it’s apt to use LLMs to fight LLMs and call it progress. I wish I was kidding…

I totally see where you guys are coming from.

It sometimes feels silly to me to have AI reviewing AI reviewing AI all the way down - see my above comment https://news.ycombinator.com/item?id=48095831

But with human judgment inserted into the right steps, it's really LLMs leveraging human thought at key stages and then, to your point, LLMs fighting LLMs fighting LLMs all the way down until...

Someone like me who has loved software his whole life and never been able to build anything more than a front-end website himself is building entire applications. So maybe it's worth the complexity!

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#36

Post by adamthegoalie »

nkmnz wrote: post #10

Great project! I’ve build something similar, not very clean and polished, but focussed around deterministic orchestration of multiple agents via typescript, because a coordinating agent was notoriously bad at things such as fetching relevant tickets and other context. One thing I struggle with so far, though, are the actual instructions for the review themselves. They are either too vague, leading to superficial or o…

That's awesome to hear and I'd love to see it when you're ready.

I actually think having something like adamsreview orchestrated by deterministic code - instead of simply having AI agents use deterministic code occasionally as this app does - could be even better!

The problem I ran into is that if you build a deterministic app that happens to use LLMs instead of the other way around, I don't think there's any way to get it to use your Claude Code subscription credits. It has to use API. And something like adamsreview would end up being so expensive if not subsidized by Anthropic along with the rest of our CC usage.

Curious to hear about your experience.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#37

Post by adamthegoalie »

stingraycharles wrote: post #4

Holy vibe coding batman this looks like a repository with just a bazillion prompts of which there are already a million. Seems like it would create a lot of friction and burn a lot of tokens.

It burns a lot of tokens, that's for sure.

Friction - maybe? Depending on what you mean.

But it's extremely useful and effective compared to everything else out there that I've tried, if you're looking for an AI code review. Let me know if you try it - or find anything else that might work too without the bazillion prompts :)

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#38

Post by adamthegoalie »

esafak wrote: post #2

That's looks like a fair bit of ceremony for what it does. Is this representative of the output? https://github.com/adamjgmiller/adamsreview/pull/3

No not at all! I haven't used it to review itself actually, for the most part, because adamsreview is mostly english language, not much code.

Here's a comment from adamsreview, but even this was 3 weeks ago, and I've worked on it a lot since then: https://github.com/cdinnison/ray-finance/pull/8#issuecomment...

I'll try to find a good public PR to review sometime soon so I can share that and add it to the Readme. This is really good feedback. I should have had something like this ready before posting to Show HN.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#39

Post by adamthegoalie »

Does anyone have an open PR on a public repo?

I'll run this against your PR for you with my CC credits as a sort-of benchmark! Send me your PR link :)

I'm going to create one on one of my other repos meanwhile and add a link to the review when it's ready.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#40

Post by esperent »

lmeyerov wrote: post #28
kuri-sun wrote: post #27

Curious what kinds of bugs the multi-agent setup catches thatsingle-pass review misses in practice. Is it more about coverage(different agents looking at different aspects) or about getting a second opinion on the same aspect? The README has examples but the mechanism by which the parallelism actually helps isn't obvious to me from them.

Yes, being comprehensive, so early or blatant cheapo findings do not distract from other ones. That's important for base results. Splitting in both file and task is (currently) important. Additionally, we run in a loop until it stops finding things, and as part of that, do test amplification when it does find any. We regularly see 3-8 rounds yielding valid results. IMO half the value is customization to your repo, so…

How do you prevent it from increasing scope?

That's the main issue I've found from running loops like this. Each loop has ~7 agents, say, looking through different lenses (security, UX, performance, etc.). Each one notes a few issues, each issue gets fixed, you do 5 to 8 loops, as you say. Each individual item that gets fixed looks minor but when you add it all up at the end you've increased PR size and scope significantly.

Post reply on HN