Writing Code Was Never the Bottleneck

No one is hiring though for reading code. I have read 10 million lines of code, roughly, I have written not one line.

Now I have produced a lot of programs, just by reading them.

People should also learn how to read programs. Most open source code is atrocious, corporate code is usually even worse, but not always.

As Donald Knuth once said, code is meant to be read. The time of literate programming is gonna come at some point, either in 100 years or in 3 years.

Investors generally don't care about the actual utility of what gets built. They want a high return on investment.

I've spent the last few weeks writing a non-trivial distributed system using Codex (OpenAI's agentic coding system). I started by writing a design brief, and iterated with o3 to refine it so it was more complete and less ambiguous. Then I asked it to write a spec of all the messages - didn't like its first attempt, but iterated on it til I did like it. Then got it to write a project plan, and iterated on that. Only then did I start on the code. The purpose of all this is to provide it some context.

It generated around 13K lines of Go for me in just over two weeks. I didn't previously speak Go, but its not hard to skimread to get the gist of its approach. I probably wrote about 100 lines, though I added and removed a lot of logging at various times to understand what was actually happening. I got it to write a lot of unit tests, so that coverage testing is very good. But I didn't actually pay a lot of attention to most of those tests on the first pass, because it generally got all the fine detail stuff exactly right on the first pass. So why all the tests? First, if something seems off, I have a place to start a deep dive. Second, it pins down the architecture so that functionality can't creep without me noticing that it is needing to change the unit tests.

Some observations.

- Coding this way is very effective - the new models almost never make fine detail mistakes. But I want to step it through chunks of new functionality at a size that I can at least skim and understand. So that 13K LoC is about 300 PRs. Otherwise I lose track of the big picture, and in this world, the big picture is my task.

- Normally the big design decisions are separated by days of fine detail coding. Using codex means I get to make all those decisions nearly back-to-back. This is both good and bad. The experience is quite intense - mostly I found the fine-detail coding to be "therapeutic", but I don't get that anymore. But not needing to pay attention to the fine detail (at least most of the time), means I think I have a better picture in my head of the overall code structure. We only have so much attention at any time, and if I don't have to hold the details, I can pay attention to the more important things.

- It's very good at writing integration tests quickly, so I write a lot more of them. These I do pay a lot of attention to. Its these tests that tell me if I got the design right, and if not, these are the place I start digging to understand what I need to change.

- Because it takes 10-30m to come back with a response, I try to keep it working on around three tasks at a time. That takes some effort, as it does require come context switching, and effort to give it tasks that won't result in large merge conflicts. If it was faster, I would not bother to set multiple tasks in parallel.

- Codex allows you to ask for multiple solutions. For simpler stuff, I've found asking for one is fine. For slightly more open questions, it's good to ask for multiple solutions, review them and decide which you prefer.

- Just prompting it with "find a bug and suggest a fix" every now and then often shows up real bugs. Mostly they tend to be some form if internal inconsistency, where I'd changed my mind about part of the code, and the something elsewhere needed to be changed to be consistent.

- I learned a lot about Go from it. If I'd been writing myself, my Go would have looked more like C++ which I'm very familiar with. But it wrote more idiomatic Go from the start, and I've learned along the way.

- Any stock algorithm stuff it will one-shot. "Load this set of network links, build a graph from them, run dijkstra over the graph from this node, and tell me the histogram of how many equal-cost shortest paths there are to every other node." That sort of stuff it will one-shot.

- It's much better than me about reasoning about concurrency. Though of course this is also one of Go's strengths.

Now I don't have any experience of how good it would be for maintaining a much larger codebase, but for this sort of scale of utility, I'm very impressed with how effective it has been.

Disclaimer: I work at OpenAI, but on networks, not AI.

I think this is a good use of AI. Change your thinking - the code is, and has always been, a medium between the computer and the human. Where is the human? Where do we define our intent? AI gives us a chance to redefine that relationship or at least make it more fluid.

A well-architected system is easier to develop and easier to maintain. It makes sense to put all the human effort into producing that because, lo and behold, both humans and LLMs can produce much better results within a well-defined structure.

> There was also a review of that code about a week later [0] which highlights the problems with LLM-generated code.

Not really. This "review" was stretching to find things to criticize in the code, and exaggerated the issues he found. I responded to some of it: https://news.ycombinator.com/item?id=44217254

Unfortunately I think a lot of people commenting on this topic come in with a conclusion they want to reach. It's hard to find people who are objectively looking at the evidence and drawing conclusions with an open mind.