I have been a software engineer for 25 years and it's my life's dream job. I have started to code with LLMs extensively as copilot's beta happened 3 years ago. There is no going back.
It's (still) easy to dismiss them as "they're great for one offs", but I have been building "professional" code leveraging LLMs as more than magic autocomplete for 2 years now and it absolutely works for large professional codebases. We are in the infancy of discovering patterns on how to apply statistical models to not just larger and more complex pieces of code, but to the entire process of engineering software. Have you ever taken a software architecture brainstorming session's transcript and asked Claude to convert it into github tickets? Then to output those github tickets into YAML? Then write a script to submit those tickets to the github API? Then to make it a webserver? Then to add a google drive integration? Then to add a slack notification system? And within an hour or two (once you are experienced), you can have a fully automated transcript to github system. Will there be AI slop here and there? Sure, I don't mind spending 10 minutes cleaning up a few tickets here and there, and spend the rest of the time talking some more about architecture.
The thing that I think many people are not seeing is that coding with LLMs at scale turns well-established software engineering techniques up to 11. To have an LLM do things at scale, you need to have concise and clear documentation / reference / tutorials, that need to be up to date always, so that you can fit the entire knowledge needed to execute a task into an LLM's context window. You need to have consistent APIs that make it hard to do the wrong thing, in fact they need to be so self-evident that the code almost writes itself, because... that's what you want. You want linting with clear error messages, because feeding those back into the LLM often helps it fix small mistakes. You want unit tests and tooling to the wazoo, structured logging, all with the goal of feeding it back into the LLM. That these practices are exactly what is needed for humans too is because... LLMs are trained on the human language we use to communicate with machines.
When I approach coding with LLMs, I always think of the humans involved first, and targeting documents that would be useful for them is most certainly going to be the best way to create the documents relevant to the AI. And if a certain task is indeed too much for the LLM, then I still have great documents for humans.
Let's assume we have some dirty legacy microservice with some nasty JS, some half baked cloudformation, a dirty database schema and haphazard logging. I would:
- claude, please make an architecture overview + mermaid diagram out of this cloudformation, the output of these aws CLI commands and a screenshot of my AWS console
- spend some time cleaning up the slop
- claude, please take ARCHITECTURE.md and make a terraform module, refactor as needed (claude slaps at these kind of ~1kLOC tasks)
- claude, please make a tutorial on how to maintain and deploy terraform/
- claude, please create a CLI tool to display the current status of the app described in ARCHITECTURE.md
- claude, please create a sentry/datadog/honeycomb/whatever monitoring setup for the app in ARCHITECTURE.md
Or:
- claude, please make an architecture overview + mermaid diagram for this "DESCRIBE TABLES" dump out of my DB
- edit the slop
- claude, please suggest views that would make this cleaner. Regenerate + mild edits until I like what I see
- claude, please make a DBT project to maintain these views
- claude, please make a tutorial on how to install dbt and query those views
- claude, please make a dashboard to show interesting metrics for DATABASE.md
- etc...
You get the idea. This is not rocket science, this literally works today, and it's what I do for both my opensource and professional work. I wouldn't hesitate to put my output at 20-30x, whatever that means. I am able to bang out in a day software that probably would have taken me a couple of weeks before, at a level of quality (docs, tooling, etc...) I never was able to reach due to time pressure or just wanting to have a weekend, WHILE maintaining a great work life balance.
This is not an easy skill and requires practice, but it's not rocket science either. There is no question in my mind that software engineering as a profession is about to fundamentally change (I don't think the need for software will diminish, in fact this just enables so many more people who deserve great software to be able to get it). But my labor? The thing I used to do, write code? I only do it when I want to relax, generate a tutorial about something that interests me, turn off copilot, and do it for "as a hobby".
Here's the writeup of a workshop I gave earlier this year: https://github.com/go-go-golems/go-go-workshop
And a recent masto-rant turned blogpost: https://llms.scapegoat.dev/start-embracing-the-effectiveness...