Earlier quoted context omitted.
That's not (or should not be what's happening). They write a short high level plan (let's say 200 words). The plan asks the agent to write a more detailed implementation plan (written by the LLM, let's say 2000-5000 words). They read this plan and adjust as needed, even sending it to the agent for re-dos. Once the implementation plan is done, they ask the agent to write the actual code changes. Then they review that…
I take this concept and I meta-prompt it even more. I have a road map (AI generated, of course) for a side project I'm toying around with to experiment with LLM-driven development. I read the road map and I understand and approve it. Then, using some skills I found on skills.sh and slightly modified, my workflow is as such: 1. Brainstorm the next slice It suggests a few items from the road map that should be worked o…
3.5 Prove
Have the LLM demonstrate, through our current documentation and other sources of facts, that the planned action WILL work correctly, without failure. Ask it to enumerate all risks and point out how the plan mitigates each risk. I've seen on several occasions, the LLM backtrack at this step and actually come up with clever so-far unforeseen error cases.