Tips for programmers to stay ahead of generative AI

LLM-based regex generation is probably my absolute favorite thing. I give it a bunch of examples and say "create a regex of that"

Does it hallucinate? Probably! Do I "hallucinate" while trying to clobber together terrible regexes? Absolutely

It's nice that we've taught the robots to make off-by-one errors just like a real developer.

I haven't faffed about with ChatGPT yet, mainly because of the verification cans it makes me drink in order just to use it. I also don't want every query of mine being snarfed up into some cloud database to be recycled for profit by Microsoft. Writing a program is a very private activity for me and I am very concerned about leaking data about my code to some huge far-off entity to profile me with. Yes, I know that there's a difference between private-time programming and programming on the job; given the industry I work in, my employer has even bigger concerns about data leakage and recently issued a policy forbidding the use of ChatGPT and other publicly available LLMs.

I do not feel that it will affect me terribly much. I don't even use autocomplete -- too distracting. I've long been of the opinion that things like autocomplete are there to simulate the feeling of increased productivity, without making you much, if any, more productive, because the bottleneck in writing code is the deep thought about what you need to write, not actually typing it in. I felt the same way about AppWizards and other code generation tools from old-school Visual Studio and the like. They generated boilerplate code for an application in the shape that some Microsoftoid decided was best, not the shape I actually wanted to create. I suspect that in the long run, LLMs will be about the same, until we've solved AGI -- at which point any such intelligence will have its own ideas about the code it wants to write, which doesn't affect me unless I choose to collaborate with it.

If you think about a human who isn't terribly smart, but they want the world to think they are, what they will do is generate bullshit to fill in the vast gaps in their knowledge. So if you have such a person working for you, you have to check their work because they will try to fob off shitty work rather than ask for help. And ChatGPT is kinda like that: it will generate bullshit (we call it "hallucinations" in the case of GPT, but the term of art is bullshit) to fill in the gaps of what was not in its training set. And there's no way to know where the gaps are. So you have to check anything it outputs for correctness and lack of bullshit. I'm not sure that incorporating LLMs into programming is (yet) not just an infinite generator of messes for humans to clean up.

Fuck it, ship it!

No, they do not. That's not how LLMs work, and stating that it is betrays an absolute lack of understanding of the underlying mechanisms.

LLMs generate statistically likely sequences of tokens. Their statistical model is derived from huge corpora, such as the contents of the entire (easily searchable) internet, more or less. This makes it statistically likely that, given a common query, they will produce a common response. In the realm of code, this makes it likely the response will be semantically meaningful.

But the statistical model doesn't know what the code means. It can't. (And trying to use large buzzwords to convince people otherwise doesn't prove anything, for what it's worth.)

To see what I mean, just ask ChatGPT about a slightly niche area. I work in programming languages research at a university, and I can't tell you how many times I've had to address student confusion because an LLM generated authoritative-sounding semantic garbage about my domain areas. It's not just that it was wrong, but that it just makes things up in every facet of the exercise to a degree that a human simply couldn't. They don't understand things; they generate text from statistical models, and nothing more.