Dev jobs are about to get a hard reset and nobody's ready

It's useful for answering some questions about Nixpkgs conventions, but it's not much faster than just looking in the manuals or reading the source.

But when it comes to looking at my actual code and answering questions about it, it's extremely hit and miss. It's good at constructing simple functions, but the module code it writes sometimes inappropriately imports idioms from other languages. It hallucinates often.

And it's absolutely worse than useless for debugging Nix module system issues. It gives nonsense answers to questions about infinite recursion issues which manage to be plausible enough to make me waste a lot of time thinking about them, before I learned more details of the module system.

After getting burned for following it down its rabbit holes, I unfortunately find myself ignoring most of its output related to this project, even as I continue to reflexively ask it things. I have often noticed in these cases that it turns out to have been right, but am left still with the empty feeling not that I shouldn't have bothered to figure out my own answer, but that I shouldn't have bothered to ask.

All of that is to say: I think working in a language/ecosystem LLMs are "bad at" is a useful sanity check. The ways that LLMs suck at languages they suck with are instructive because they reiterate the nature of the things. The failure modes are still what you'd expect from a stochastic parrot, even as the models get "smarter".

The massive training data pools for more popular programming ecosystems make it too easy to fool yourself into believing that these things can reason. The unevenness of their performance tells you what hasn't actually "generalized".

Replacing something with something else isn't a refactor to me, a refactor implies a structural change not a wide surface level change.

Those skills require knowing how code works. You can't leapfrog into a senior dev., at least not in most cases. Languages have drastically different features plus the AI agents will not be perfect and you'll need to review code and probably modify some of it. The more you know how to code the easier this will be.

Let's park coding to the side for a bit.

Case 1:

I am collaborating with a friend to build a graded Sanskrit reader for beginners using the Aesop's fables.

As a precursor, I asked Gemini 2.5 Pro if it had access to all the stories. Yes, it said. The three popular PD ones? Yes.

I asked it to print all three versions of a particular one, and it did. One of them was not the version it confidently claimed it was. We argued about it for a while. It shut up when I provided actual evidence.

I then decided to upload the three Gutenberg text files and asked it to use them as the source of truth to give me a list of unique stories putting variant plots, variant titles etc under the main heading. I gave it certain formatting requirements so that I could later verify if all 600-odd tales across the three books were properly accounted for.

Gemini tied itself into knots trying to do this. It could not guarantee that all the tales were present in the list it generated. It didn't know how to accomplish the task. Finally, I gave it a series of steps, an algorithm based on an n-branched tree. Only then did it manage to generate the list for me.

This took me four hours of wrangling across three different sessions.

Case 2:

I have been buying TASCHEN editions of impressionists and other classical artists. I wanted Gemini to compare various editions, give me the pros and cons so that I could pick a good edition to buy. By the time we came to Michelangelo it went nuts, hallucinating editions, ISBN numbers, page counts, authoritative urls, worldcat searches ...

This took about two hours.

There are more such amusing anecdotes. Some from DeepSeek as well.

I have tried LLMs with python, and typst and a few other things. Sometimes they work, sometimes they don't. They definitely do not write code the way I want them to. They will use OOP even if I specifically warn them not to.

LLMs are VERY good at translation and languages. I will give them that. But reasoning? I am not convinced. I will believe that LLMs are good enough to replace programmers when the Amodei siblings can operate their company only using LLM developers.