Is this the end of human code review?

This title doesn’t match the article.

Think we're pretty close to the point where we'll no longer need to prompt AI to refactor or review the refactoring if it's done within tight constraints i.e. absolutely no change to functionality. But when functionality changes may be acceptable as part of it, it's less clear - in my experience AI isn't at a point yet where it can always be trusted to "understand" what the requirements are (including to what extent they can be allowed to evolve), what is or isn't acceptable in terms of user experience etc. That's where human review is still useful and in many cases, essential.

"Here's a case study where it worked once" != "the end of human code review".

Don't get me wrong, this is impressive and capabilities do still seem to be on a generally upward trend with nothing "hitting a wall" despite all the parroting of that phrase, but there's a huge gap between the first time a machine manages something impressive enough to document, and that machine becoming so good at that task that humans need not apply.

The codebase is a VS Code extension implementing an AI coding agent.

*The main prompt:*

The request was submitted to the agent as follows, in a single natural-language prompt with no accompanying formal specification:

"Analyze this request in detail. Produce a comprehensive report covering understanding and feasibility.

- The system that streams an active session on the frontend within EasyAgents: What is it? A complex system that distinguishes between current and past messages; it is tricky to manage, involves handling streaming pauses, includes mechanisms to prevent data loss due to race conditions, etc. It requires detailed study, it is extremely complex and full of pitfalls, not something to be looked at casually. We will start by analyzing it closely, then extract the code and create technical documentation for it within a new dedicated module (likely located somewhere under`orch/`). "Extract" means moving the code while making it generic, ready for use in both the EasyAgents use case and the standard session use case. The extraction must be done properly, including all associated systems, no sloppy coding. A successful extraction at this stage means the code is separated and reconnected to EasyAgents without EasyAgents actually containing the code itself; the visible functionality remains the same (the change is invisible on screen but real in the codebase).

- What needs to be done? Extract it from EasyAgents, move it, and make it generic.

- What is the goal? It must work with standard AI-code sessions.

- What is the result? Closing a standard AI-code session view no longer kills the orchestrator process; the session remains visible in the activity sidebar.

- Why? We sometimes encounter a bug that turns the view gray (an Electron memory overflow issue in VS Code), which is annoying because nothing is visible. Currently, we have to let it finish blindly until the activity view indicates completion, or kill the request, but that means losing the work done and having to restart the last user message from scratch. With this new system, if the bug occurs, we can close the view and reopen it by clicking on it in the activity list. Reopening it restores the display.

- UX adaptation required: Since the process isn't killed when the view closes, we need a way to kill it. In the activity view, each item currently has two lines (title and elapsed time). Add a third line containing a stop button (stop icon, small size, danger style). This button should appear only when the view is hidden but the process is still running; it (and the third line) should not appear if the process finishes normally or if the view is displayed again. The button becomes disabled after being clicked to prevent accidental double-clicks and bugs. Clicking it kills the process, just as closing the window currently does.

From my reading of this it seems that the bug they were fixing was that there was a chat system which wasn't originally designed to be able to recover from a reload as it didn't track the state of both clients and didn't have any way for clients to catch up. The fix cost $2,430, took 3 days and touched ~50k lines, 189 files (7.1% of the codebase by LoC)

I totally agree that with the right systems we can do away with code review, and it's a solidly impressive achievement by the model/harness. But three days an nearly $2,500 to fix something that is a very foreseeable requirement if you'd done a little bit of planning doesn't seem like a terribly impressive outcome.

I think the more interesting tradeoff here is that the system they had to fix feels like it was "vibed" rather than "engineered" (or maybe the lack of state tracking was a deliberate tradeoff, but it doesn't really read that way) it sounds like they're at the "find out" stage TBH. But maybe spending $2,500 to fix things that would have been easily solved by some forethought is a tradeoff worth making if you're shipping things at ludicrous speed. Maybe you just ship 10 features and only fix the ones that get traction and it works...?