Some uncomfortable truths about AI coding agents

> In reality, though, the code review load for software engineers will gradually increase as fewer and fewer of them are expected to supervise an ever-growing number of coding agents, and they will inevitably learn to become complacent over time, out of pure necessity for their sanity. I’m a proponent of code review...but even I often consider it a slog to do my due diligence for a large code review (just because I think it’s important doesn’t mean I think it’s fun). If it’s your full-time job to review a swarm of agents’ work, and experience tells you they are good enough 95%+ of the time, you’re not going to pay as much attention as you should and bad changes will get through.

Another way to look at this is that AI coding agents take the fun out of a software engineer's job. The machine takes many of the fun parts and leaves the human with more of the unenjoyable parts.

Under our new ways of working, you are required to be excited an curious about this evolution three times per day.

> Skill atrophy

If LLMs are so good that you no longer have use for the skill, why do we care about skill atrophy? That skill isn't that useful to most people. There are so many examples of this in human history where it was completely fine and we went on to do higher order things that were more useful.

> Even if they set out fully intending to provide the highest level of scrutiny to all generated code, they will gradually lose the ability to tell a good change from a bad one

If this (first order effect) is actually a problem then it follows that we will naturally exercise our skill of detecting good change from bad ones (second order effect) and the skill will not atrophy? (third order effect). Seems like your "problem" is self correcting?

> At its core, the only defense I’ve got for that response is… this time feels different? Not a particularly rigorous defense, I admit, but I did warn you that this was the squishiest of the issues at hand.

Well, if you knew this perhaps it was better just not to lead with it and spend so many paragraphs on it.

> Some might argue that, even if that time comes eventually, that’s no reason not to make use of the tools that are available right now. But it should come as no surprise that I disagree. Better not to become overly dependent on AI coding agents in the first place so you’ll be better situated to weather the storm (and maybe even thrive) when it comes.

Well this argument didn't turn out to be any less squishy than the first one. It's a self correcting "problem" but you disagree and we should do X because you said so. What was the point of all of this then?

> Prompt Injection

I also think this will likely always be a problem but you can pretty much point at ANY tool we use in software development. Your viewpoint would be similar to saying we should stop using libraries because there's always going to be a vulnerability when you distribute code that somewhere in the chain a bad actor can inject malicious code even if the library was created by a trusted source in the industry. We have plenty of examples of this happening in real life. So far, still squishy.

> Copyright/licensing > I’m not a lawyer! I’m a legal layperson offering my unqualified assessment of some tricky legal questions. Let’s get to it.

Sigh, this entire post is slop isn't it? Bad look for whatever "standup for me is".

edit: Standup for me is something that is made entirely irrelevant by agentic LLMs, no surprise. The irony is rich.

The author wants to be the gatekeeper of skill, quality, and how we develop while they hand feed us slop in the form of their blog posts.

yeah there is only one surefire 100% fix for “prompt injection”: use deterministic solutions ie not machine learning.

----

addendum in case someone tries to make this commonly made point -- i don't use deterministic here to mean "i've pinned the ML model weights after training". i use it in reference to the probability theory stuff of training/models (the boring and complicated maths stuff).