AI Is Making Developers Dumb

I use LLMs to write entire test functions, but I also have specs for it to work from and can go over what it wrote and verify it. I never blindly go "yeah this test is good" after it generates it.

I think that's the middle ground, knowing where, and when it can handle a full function / impl vs a single/multi(short) line auto-completion.

A junior developer was tasked with writing a script that would produce a list of branches that haven't been touched for a while. I've got the review request. The big chunk of it was written in awk -- even though many awk scripts are one-liners, they don't have to be -- and that chunk was kinda impressive, making some clever use of associative arrays, auto-vivification, and more pretty advanced awk stuff. In fact, it was actually longer than any awk that I have ever written.

When I asked them, "where did you learn awk?", they were taken by surprise -- "where did I learn what?"

Turns out they just fed the task definition to some LLM and copied the answer to the pull request.

The problem is that the LLM won't find design mistakes. E.g. trying to get the value of a label in Textual, you can technically do it, but you're not really suppose to. The variable starts with an underscore, so that's an indication that you shouldn't really touch it. The LLMs will happily help you attempt to use a non-existing .text attribute, then start running circles, because what you're doing is a design mistake.

LLMs a probably fairly helpful for situations where the documentation is lacking, but simple auto-complete is also working well enough.