The End of Programming

I would love it if it was that simple :-) Agents can one-shot this prototype and that feels like they are "almost done". The hard part starts when we try to make it production-grade.

I'm not saying that it's not possible, but it requires more effort than productionizing a PoC that we used to create "by hand". We start to discover shortcuts taken by the coding agent and we spend 80% of the time on getting the last 20% right (but to be clear, it's still less time than writing it by hand).

It looks like the author may not have hit this reality check yet:

> Of course, neither of these things is currently shipped, or supported and isn’t what I’d call production ready software. So you could say what many say about AI, which is that it helps you ship the prototype faster. But that isn’t really giving enough credit here.

For AI-generated code, finding "the last 20% rough edges" is much, much harder than creating PoC. Often the rough edges are buried in the code that we don't know. Fixing those problems requires big refactors that, for wrongly architected code, can break other things.

Tests help, but that's not enough for models to autonomously fix them. If they make one error, it compounds over multiple iterations.

It's especially true for problems where we don't have a clear oracle. It's impressive that models can brute-force problems with a clear oracle. But many problems where a clear oracle doesn't exist (a lot of business software that I know) will still require a lot of product engineering.

Maybe it will improve? I'm not sure if it can happen with hallucinations around (from what I see, they are still a big deal for domains that agents are not trained on).

Exactly, tight E2E test coverage and obviously manual functional verification to seal the coverage gaps will be needed even more.

1. The resulting code was of pretty low quality. Others that bothered to put it through Miri and the like found many soundness issues, but my personal favorite example is this example which is trivially and locally (meaning that someone who has the most basic understanding of unsafe in rust can see it's obviously wrong just by looking at the specific function) incorrect example [0]. This particular example was removed in an apparently unrelated refactor after spending well over a month in the code base without any of the bun maintainers or their agents detecting it, and a quick grep found hundreds of potential similar issues (although many of those are false positives).

2. More generally, it's not clear to me that there was any technical benefit to the rewrite in the first place. The stated reason was for memory safety, but replacing Zig with unsafe rust doesn't actually get you memory safety, and removing the unsafe blocks often requires more extensive refactors to fit within rust's model.

> If you can reduce a problem to a clearly verifiable end state, provide the necessary context, and equip a model with the necessary tools it can usually get to a good solution.

As others have pointed out (and you acknowledge), "reducing a problem to a clearly verifiable end state" is just "programming". What you don't seem to understand is that actually doing that is made harder by using AI, not easier. A sufficiently detailed spec is called "code" [1], the question is what language/notation is best to write it in. The answer is almost never "whatever is closest to what the computer actually executes", as assemblers and later compilers and interpreters demonstrated. But it also isn't several of the things that AI proponents have suggested to replace the latter with.

Take natural language, for example. As Dijkstra pointed out, we've been through this already with math. It used to be that all math was expressed in a way closer to what we'd now call "word problems", but this turned out to be bad. The specialized language of e.g. algebra isn't something mathematicians use to gate-keep, it's way easier to reason in the domain that way than it is in English (or other natural languages). The same is true for programming, once you actually specify what you want to do with enough rigor. It's generally easier to read and reason about code than to do so with natural language specifications.

Another proposal is to use tests and similar automatic verification to specify the program. I suspect that anyone with much experience can already tell whether it's preferable to specify a program through code or through tests, but thankfully we have empirical evidence on this for anyone who has any doubts in the form of e.g. sqlite. Sqlite is probably one of if not the closest any piece of software comes to being fully specified by it's tests. To do that takes almost 600 times as much test code as there is "regular" code. Dr. Hipp even personally weighed in on the implications this has on AI recently [3] . The reason to do testing is that it provides a second independent check for correctness, if you're using it as the *only* check that advantage disappears.

[0] https://github.com/oven-sh/bun/blob/fc865b398e51de8a95ddde4b...

[1] https://haskellforall.com/2026/03/a-sufficiently-detailed-sp...

[2] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

[3] https://youtu.be/V_qzqY1bb7I?t=2727

I'd be very surprised if there was not a lot more human work than they are making out. They're certainly not a trustworthy source.

If not, do we really think no one at Bun now has a mental model of how their own software works, and only Claude can fix it moving forward?