Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: adamsreview – better multi-agent PR reviews for Claude Code

github.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#41

Post by esperent »

adamthegoalie wrote: post #30
madamelic wrote: post #23

Neat idea. I am more curious about your AI workflow as I stay away from other's tools because I don't trust vibe-code related tools. What is the workflow difference between `fragments/` and `plans/`. They seem logically the same but seem to have been used for different purposes. Is this something it did on its own or is this something you prompted it to do?

Hey thanks for the comment and the question. I would say my workflow for any meaningful amount of work is (all in Claude Code): - PRD: I discuss and brainstorm with Claude Code using something like the Grill Me skill https://github.com/mattpocock/skills/tree/main/skills/produc... but that I've modified a bit for my own style, until I have a good PRD (what the goals / design decisions are for what I'm building) --- I…

I think there's no harm in that. But I will say, it sounds very similar to my own system, and probably a ton of other people's.

Yours might very well be better than most, but the thing that's missing from all of these is evals. You, me, everyone else, we're all vibe coding up these loops, we're getting work done and feeling excited about it, excited enough that we want to share, but nobody is doing real testing or benchmarks.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#42

Post by adamthegoalie »

esperent wrote: post #40
lmeyerov wrote: post #28

Earlier quoted context omitted.

Yes, being comprehensive, so early or blatant cheapo findings do not distract from other ones. That's important for base results. Splitting in both file and task is (currently) important. Additionally, we run in a loop until it stops finding things, and as part of that, do test amplification when it does find any. We regularly see 3-8 rounds yielding valid results. IMO half the value is customization to your repo, so…

How do you prevent it from increasing scope? That's the main issue I've found from running loops like this. Each loop has ~7 agents, say, looking through different lenses (security, UX, performance, etc.). Each one notes a few issues, each issue gets fixed, you do 5 to 8 loops, as you say. Each individual item that gets fixed looks minor but when you add it all up at the end you've increased PR size and scope signifi…

That is such a good point.

I recently opened a PR against this AI personal finance tool Ray https://github.com/cdinnison/ray-finance/pull/8 to add an Apple Card import feature, since Apple Card is not supported by Plaid.

I built the manual import feature, opened the PR, and then ran a code review.

What I hadn't thought about when I built the feature, was the myriad ways that the implications of importing data from Apple would have to be considered and integrated into the rest of the app, for the manual import to be a first-class feature, not "just a manual import" of data.

I ended up running adamsreview against it like 5-10 times, before considering it complete, as I learned that there was much more to the integration than I realized.

Now is that necessarily a problem? Maybe not. I should have realized from the start that the import feature was going to much more than just a small feature. But at least, thanks to the review loop, I got it completely right before the PR was merged.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#43

Post by adamthegoalie »

esperent wrote: post #41
adamthegoalie wrote: post #30

Earlier quoted context omitted.

Hey thanks for the comment and the question. I would say my workflow for any meaningful amount of work is (all in Claude Code): - PRD: I discuss and brainstorm with Claude Code using something like the Grill Me skill https://github.com/mattpocock/skills/tree/main/skills/produc... but that I've modified a bit for my own style, until I have a good PRD (what the goals / design decisions are for what I'm building) --- I…

I think there's no harm in that. But I will say, it sounds very similar to my own system, and probably a ton of other people's. Yours might very well be better than most, but the thing that's missing from all of these is evals. You, me, everyone else, we're all vibe coding up these loops, we're getting work done and feeling excited about it, excited enough that we want to share, but nobody is doing real testing or be…

That is such a great point. We do need evals for this - and not just ones that the model companies use themselves. They have to be public and sharable and easy to use ourselves.

And in terms of sharing, I agree. On one hand, so many of us are already doing this themselves. On the other hand, when I was first learning CC and agentic engineering (vibe coding at the time :) ), I did find some of these random people's templates useful.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#45

Post by psadauskas »

yuppiepuppie wrote: post #7

The best code review improvement I have done in my workflow with Claude is using tuicr ( https://tuicr.dev ). It runs locally, YOU review all the code locally, and feedback that to Claude. Agents reviewing AI code always felt dirty to me, especially when working on production (non-disposable) code.

I've been using https://github.com/choplin/code-review.nvim, which looks like a similar UI, but in the NeoVim interface. `rc` to comment on a line/selection, then `ry` to yank all comments into the clipboard to be pasted into a chat.

It leaves the comments as markdown files in ./.code-review, so I also have my `/review` agent set to output in the same format, so an LLM can be reviewing the same code I am, I can edit or dismiss the LLM's reviews, then send the whole thing back to the first agent to fix.

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#46

Post by adamthegoalie »

adamthegoalie wrote: post #39

Does anyone have an open PR on a public repo? I'll run this against your PR for you with my CC credits as a sort-of benchmark! Send me your PR link :) I'm going to create one on one of my other repos meanwhile and add a link to the review when it's ready.

Meanwhile here's what it looks like - just ran it on my own other repo: https://github.com/adamjgmiller/worktreehq/pull/145#issuecom...

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#47

Post by adamthegoalie »

esafak wrote: post #2

That's looks like a fair bit of ceremony for what it does. Is this representative of the output? https://github.com/adamjgmiller/adamsreview/pull/3

Here's an example: https://github.com/adamjgmiller/worktreehq/pull/145#issuecom...

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#48

Post by phonon »

adamthegoalie wrote: post #31
parasmadan wrote: post #18

Why not just use an eval harness to prove this catches more real bugs? Benchmarks on actual bug classes would be far more convincing than comparing against /review.

That's a great idea. I had trouble finding anything like this, a benchmark made for (AI) code reviewers. I had expected to find something like an eval harness available on GitHub, but couldn't find it. Any suggestions? Or maybe we/I/someone should build something like this? I suppose one challenge is that if it's going to be publicly available, it would also be easy to cheat, but still seems it would be useful if peo…

https://www.codereviewbench.com/

https://codereview.withmartian.com/

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#49

Post by esafak »

adamthegoalie wrote: post #29
kuri-sun wrote: post #27

Curious what kinds of bugs the multi-agent setup catches thatsingle-pass review misses in practice. Is it more about coverage(different agents looking at different aspects) or about getting a second opinion on the same aspect? The README has examples but the mechanism by which the parallelism actually helps isn't obvious to me from them.

I was thinking about building a GitHub repo made for evaluating Code Reviews. Something like a complex app (or perhaps a few branches with different options), and then PRs on each branch with varying types and degrees of bugs for a Code Review to find. I suppose this would not be a 'real' benchmark because it would be public and so you couldn't necessarily trust scores people share about how their own tool did, but i…

https://codereview.withmartian.com/

https://www.greptile.com/benchmarks

Re: Show HN: adamsreview – better multi-agent PR reviews for Claude Code

#50

Post by tomasz-tomczyk »

yuppiepuppie wrote: post #7

The best code review improvement I have done in my workflow with Claude is using tuicr ( https://tuicr.dev ). It runs locally, YOU review all the code locally, and feedback that to Claude. Agents reviewing AI code always felt dirty to me, especially when working on production (non-disposable) code.

Same experience here - I built a similar tool, for reviewing both plans and code - https://crit.md (shameless plug), browser based as opposed to TUI.

Having said that, I don't review the code until going through a few iterations of reviews from Claude. Each round it does find some "obvious" issue, so as long as I'm not close to maxing my subscription for the week I let it run an audit -> validator checks the claims -> fix issues before I get to it.

Post reply on HN