Live data from Hacker News

Orchestrating AI code review at scale

blog.cloudflare.com

11–20 of 66 posts

Re: Orchestrating AI code review at scale

#11
post #2

> The entire system also runs locally. I think approaches like this don't need to run other than locally. Maybe integrated as pre-push hook. The system is nondeterministic, so it's at odds with the purpose of CI.

I'm not sure the people integrating it into CI process understand what CI is.

Same can be said about human review if the argument is non-determinism.

Re: Orchestrating AI code review at scale

#13
post #8

> Today, when an engineer at Cloudflare opens a merge request, it gets an initial pass from a coordinated smörgåsbord of AI agents. I’d prefer to have that happen as some sort of pre commit hook, before a merge request is sent. The feedback loop might be a bit faster and the process might produce less noise this way.

My company has the AI review agents, and you can run them locally, but practically it’s easier to just open a merge request to have CI run the agents. Especially if you’re juggling a bunch of merge requests.

Re: Orchestrating AI code review at scale

#14
post #7

> One of the operational headaches we didn’t predict was that large, advanced models like Claude Opus 4.7 or GPT-5.4 can sometimes spend quite a while thinking through a problem, and to our users this can make it look exactly like a hung job. I had the same problem in my recursive agent harness. It would always come back, but it could sometimes take up to 10 minutes depending. I fixed this by adding a required "purpo…

Nice trick! I am doing something similar but passing those incremental updates to Haiku for a short user-friendly message.

Re: Orchestrating AI code review at scale

#17
post #8

> Today, when an engineer at Cloudflare opens a merge request, it gets an initial pass from a coordinated smörgåsbord of AI agents. I’d prefer to have that happen as some sort of pre commit hook, before a merge request is sent. The feedback loop might be a bit faster and the process might produce less noise this way.

Valid, but you lose the lived history that comes with the audit log of it being actual review back and forth and CI runs vs lost to a developers machine and only a relic in the commit log. I can see both sides, though.

Can you elaborate about the practical value of having the history of back and forth, in a PR or even in the commit log? In my 20ish years of experience, I can’t recall a single instance where I’ve solved something thanks to having this work-in-progress state persisted in the repo history.

It’s exclusively been the other way around where having a smaller number of larger squished commits (post merge) that’s made the project be more maintainable.

Re: Orchestrating AI code review at scale

#18
Every iteration something can be found. How many times do you iterate e.g. on performance - use optimized struct, oh, you can change the architecture etc.? At that point one can just have a while loop for the agents to make changes until no comments left.

Re: Orchestrating AI code review at scale

#19
post #11

Earlier quoted context omitted.

I'm not sure the people integrating it into CI process understand what CI is.

Same can be said about human review if the argument is non-determinism.

Human review is about learning and there's an implied social contract in that someone is giving you their time to make you better. It isn't necessarily necessary but replacing it with AI shows a fundamental misunderstanding of why it is part of the process.

Re: Orchestrating AI code review at scale

#20
post #8

> Today, when an engineer at Cloudflare opens a merge request, it gets an initial pass from a coordinated smörgåsbord of AI agents. I’d prefer to have that happen as some sort of pre commit hook, before a merge request is sent. The feedback loop might be a bit faster and the process might produce less noise this way.

I have mixed feelings, but it boils down to how long it takes and / or cost.

Pre-commit hooks should be fast, as it's something you'd do (normally) a few dozen times a year. I don't believe sending a review job to a remote agent is fast, nor will waiting for a review to finish a commit be good for anyone.

CI on the other hand can be slower and runs async, it's fire and forget so you can switch tasks.

If noise is an issue, one possible solution is to create a merge request, have the tools review it, make the fixes, rewrite history like you did it perfectly the first time ("fix" commits are noise), then create a new one for human review.

Post reply on HN