Ask HN: How would you harden AI changes to a 1M-line legacy SaaS before review?

I’m not a software engineer, but I’ve been running an experiment to see whether agentic development could produce a useful prototype on top of an existing SaaS codebase.

The codebase is 1M+ lines, 15 years old, hosted on Azure, and primarily written in C# and React.

The prototype needs to be available for customer testing in September. No developers were available to work on it full-time, although I could occasionally get help with specific technical issues. An engineer will evaluate the implementation in August and decide how much confidence we can have in the AI-generated code so that we can decide how to “convert” the code to production-grade.

My question is what I can do before August, primarily using AI tools and manual testing, to make the code as robust and reviewable as possible. I want to increase the likelihood that the AI-generated code would be so good that the path to production-grade would be closer to “copy-paste” than building everything again from scratch.

Environment The prototype is being developed in a separate branch, deployed to a separate internal environment, and connected to its own database and schema.

Planning process We interviewed customers in June and turned the resuls into an MVP spec.

The process was approximately: 1. Write a PRD. 2. Use an LLM to convert the PRD into an architecture document, which was reviewed by an architect. 3. Create product designs consisting of screen images and md files containing interaction details ans edge cases with Claude Design. 4. Use an agent to break the work into epics using the PRD and architecture document as guardrails.

The epics were the most granular planning artifacts that received review by me and the architect.

Development process The development flow was intended to run with little intervention: 1. A planner agent converted epics into md story files and Jira stories 2. A coding agent implemented the stories including tests and opened PRs 3. A reviewer agent reviewed the PRs, requested changes, and merged them into the prototype branch

The coding agent polled PRs for review comments and could escalate issues back to the planner.

Agents also had a “stop and ask” list for decisions they were not allowed to make autonomously. I (and a few times an engineer) were involved by resolving those escalations and by manually testing the accumulated changes end to end each day.

Most implementation and initial review were done with Claude-based agents. For riskier PRs, I also used Codex as a reviewer. The second-model review found substantially more relevant issues in the Claude-generated code, but token quotas limited the use.

I had separate refactoring and harden runs for the code as well.

Results The planning and setting up the environment and agentic flow took about two weeks and then the agents built the whole MVP in about two weeks. Size-wise it was 13k lines of functional code + the same amount for tests.

What I would like advice on Assuming that I cannot get substantial developer involvement before the review, how can I increase the likelihood that the code is as close to production-grade as possible?

Here are some of the questions I have been thinking about: 1. What checks or development loops would give the largest improvement in confidence before an engineer reviews the code? 2. How would you use independent agents or models to reduce the risk that the coder and reviewer make the same incorrect assumptions? 3. Should tests be generated by a separate agent from the one that wrote the implementation? 4. What documentation or evidence would make the eventual engineering review faster and more reliable? 5. If you had only a few weeks to improve this prototype before handing it to an engineer, what would you prioritize?

At this point what stops you from asking AI this question? But honestly, extensive QA. There are probably more quirks than you could imagine

I have asked the AI, but was hoping for a second opinion, ideally by someone who has experience from doing something a bit similar. Your point about QA is interesting. Bob Martin, doesn't review AI code any more, just does extensive QA (https://x.com/unclebobmartin/status/2080257779395154409). Perhaps the key move would be for me to now focus on QA and try to nudge the engineering review to also emphasize testing over reading code?

Once you no longer "own" the code, does it matter? If you provide a very detail set of specifications and success criteria then you can turn it into a black box. The code is no longer the asset, just the business that uses it. You can use AI to extract the requirements from the existing code and then modify as needed however, I would recommend doing this in two phases: first the conversion, then the enhancements.

I suppose you are right at least what comes to the functionality. Security is the one thing where I am sure that my company will require some peaking under the hood.