How I program with LLMs

> There are three ways I use LLMs in my day-to-day programming: 1/ Autocomplete 2/ Search 3/ Chat-driven programming

I do mostly 2/ Search, which is like a personalized Stack Overflow and sometimes feels incredible. You can ask a general question about a specific problem and then dive into some specific point to make sure you understand every part clearly. This works best for things one doesn't know enough about, but has a general idea of how the solution should sound or what it should do. Or, copy-pasting error messages from tools like Docker and have the LLM debug it for you really feels like magic.

For some reason I have always disliked autocomplete anywhere, so I don't do that.

The third way, chat-driven programming, is more difficult, because the code generated by LLMs can be large, and can also be wrong. LLMs are too eager to help, and they will try to find a solution even if there isn't one, and will invent it if necessary. Telling them in the prompt to say "I don't know" or "it's impossible" if need be, can help.

But, like the author says, it's very helpful to get started on something.

> That is why I still use an LLM via a web browser, because I want a blank slate on which to craft a well-contained request

That's also what I do. I wouldn't like having something in the IDE trying to second guess what I write or suddenly absorbing everything into context and coming up with answers that it thinks make a lot of sense but actually don't.

But the main benefit is, like the author says, that it lets one start afresh with every new question or problem, and save focused threads on specific topics.

My main usage is in helping me approach domains and tools I don't know enough to confidently know how best to get started.

So one thing that doesn't get a mention in the article but is quite significant I think is the long lag of knowledge cutoff dates: looking at even the latest and greatest, there is one year or more of missing information.

I would love for someone more versed than me to tell us how best to use RAG or LoRA to get the model to answer with fully up to date knowledge on libraries, frameworks, ...

I used AI code completion from GitHub copilot on a 20 year old project. You still have to create new classes, new test, refactor etc.

> Why would you go to them rather than the actual documentation?

Not every documentation is made equal. For example: Android docs are royal shit. They cover some basic things, e.g. show a button, but good look finding esoteric Bluetooth information or package management, etc. Most of it is a mix of experimentation and historical knowledge (baggage).

I have also many years of programming experience and find myself strongly "accelerated" by LLMs when writing code. But, if you think at it, it makes sense that many seasoned programmers are using LLMs better. LLMs are a helpful tool, but also a hard-to-use tool, and in general it's fair to think that better programmers can do a better use of some assistant (human or otherwise): better understanding its strengths, identifying faster the good and bad output, providing better guidance to correct the approach...

Other than that, what correlates more strongly with the ability to use LLMs effectively is, I believe, language skills: the ability to describe problems very clearly. LLMs reply quality changes very significantly with the quality of the prompt. Experienced programmers that can also communicate effectively provide the model with many design hints, details where to focus, ..., basically escaping many local minima immediately.