Earlier quoted context omitted.
Most programmers can't make much sense of the output of a C compiler, either. We'll all be in that boat before long. (To anticipate the usual reaction when I point that out: if you're going to sputter with rage and say that compilers are deterministic while AI isn't, well... save it for a future argument with someone who can be convinced that it matters.)
on the other hand, I'd love to read an argument that persuades me that determinism doesn't matter in this case because I can't form any mental model that makes determinism-ed-ness a non factor in my decision making. of course, this comes with a disclaimer that I have no experience as a vibe coder lol
Human involvement will end after the first phase, once the required test targets exist.
Basically we are going to have to stop micromanaging our toolchains and start telling them what outcomes we want. Call it "vibe coding lol" or whatever, that's how it's going to work. When you write code in a traditional high-level language, you are telling your compiler how to generate the code that actually gets executed by the CPU. But this object code, again, might as well be completely inaccessible as far as most developers are concerned. If the compiler were to achieve the same result in different, unknowable ways each time it runs, that's a horrifying notion to those same developers... but we all might as well get used to it, because soon there will be no other way forward.
Notice that the math guys are already having to deal with similar problems. Mochizuki or somebody drops a thousand-page proof on the community, and experts at the very highest levels have to waste years looking for bugs in it. They can't go on like this and they know it. They will have to give up the quaint idea of understanding something completely in order to prove or refute it, just as programmers and engineers will have to relinquish control over how our own creations work.