Eight years of wanting, three months of building with AI

Note I believe this one because of the amount of elbow grease that went into it: 250 hours! Based on smaller projects I’ve done I’d say this post is a good model for what a significant AI-assisted systems programming project looks like.

Unlike many claims that AI works that are clearly bogus, this actually seems quite credible, because TFA describes in detail many problems encountered, which could have easily lead to a failure of the project, if not properly addressed.

There is no doubt that when used in the right way an AI coding assistant can be very helpful, but using it in the right way does not result in the fantastic productivity-increasing factors claimed by some. TFA describes a way of using AI that seems right and it also describes the temptations of using AI wrong, which must be resisted.

More important is whether the productivity improvement is worth a subscription price. Nothing that I have seen until now convinces me about this.

On the other hand, I believe that running locally a good open-weights coding assistant, so that you do not have to worry about token price or about exceeding subscription limits in a critical moment, is very worthwhile.

Unfortunately, thieves like Altman have ensured that running locally has become much more difficult than last year, due to the huge increases in the prices of DRAM and of SSDs. In January I have been forced to replace an old mini-PC, but I was forced to put in the new mini-PC only 32 GB of DDR5, the same as in the 7-year old replaced mini-PC. If I had made the upgrade a few months earlier, I would have put in it 96 GB, which would have made it much more useful. Fortunately, I also have older computers with 64 GB or 128 GB DRAM, where bigger LLMs may be run.

This essay perfectly encapsulates my own experience. My biggest frustration is that the AI is astonishingly good at making awful slop which somehow works. It’s got no taste, no concern for elegance, no eagerness for the satisfyingly terse. My job has shifted from code writer to quality control officer.

Nowhere is this more obvious in my current projects than with CRUD interface building. It will go nuts building these elaborate labyrinths and I’m sitting there baffled, bemused, foolishly hoping that THIS time it would recognise that a single SQL query is all that’s needed. It knows how to write complex SQL if you insist, but it never wants to.

But even with those frustrations, damn it is a lot faster than writing it all myself.

> More important is whether the productivity improvement is worth a subscription price. Nothing that I have seen until now convinces me about this. On the other hand, I believe that running locally a good open-weights coding assistant, so that you do not have to worry about token price or about exceeding subscription limits in a critical moment, is very worthwhile.

This is one thing I also wonder about. If it's a really good programming helper, making 20% of your job 5x faster, then you can compute the value. Say for a $250K SWE this looks like $40k/year roughly. You don't want to hand 100% of that value to the LLM providers or you've just broken even, so then maybe it is worth $200/mo.

Refreshing to see an honest and balanced take on AI coding. This is what real AI-assisted coding looks like once you get past the initial wow factor of having the AI write code that executes and does what you asked.

This experience is familiar to every serious software engineer who has used AI code gen and then reviewed the output:

> But when I reviewed the codebase in detail in late January, the downside was obvious: the codebase was complete spaghetti14. I didn’t understand large parts of the Python source extraction pipeline, functions were scattered in random files without a clear shape, and a few files had grown to several thousand lines. It was extremely fragile; it solved the immediate problem but it was never going to cope with my larger vision,

Some people never get to the part where they review the code. They go straight to their LinkedIn or blog and start writing (or having ChatGPT write) posts about how manual coding is dead and they’re done writing code by hand forever.

Some people review the code and declare it unusable garbage, then also go to their social media and post how AI coding is completely useless and they’re not going to use it for anything.

This blog post shows the journey that anyone not in one of those two vocal minorities is going through right now: A realization that AI coding tools can be a large accelerator but you need to learn how to use them correctly in your workflow and you need to remain involved in the code. It’s not as clickbaity as the extreme takes that get posted all the time. It’s a little disappointing to read the part where they said hard work was still required. It is a realistic and balanced take on the state of AI coding, though.