The hidden cost of AI coding

This to me is like every complaint I read when people generate code and the LLM spits out an error, or something stupid. It's a tool. You still have to understand software construction, and how to hold the tool.

Our Rust fly-proxy tree is about 80k (cloc) lines of code; our Go flyd tree (a Go monorepo) is 300k. Generally, I'll prompt an LLM to deal with them in stages; a first pass, with some hints, on a general question like "find the code that does XYZ"; I'll review and read the code itself, then feed that back to the LLM with questions like "summarize all the functionality of this package and how it relates to other packages" or "trace the flow of an HTTP request through all the layers of this proxy".

Generally, I'll take the results of those queries and have them saved in .txt files that I can reference in future prompts.

I think sometimes developers are demanding something close to AGI from their tooling, something that would do exactly what they would do (only, in the span of about 15 seconds). I don't believe in AGI, and so I don't expect it from my tools; I just want them to do a better job of fielding arbitrary questions (or generating arbitrary code) than grep or eglot could.

I think most of the time when we tell ourselves this, it's cope. Software is automation. "Computers" used to be people! Literally, people.

Yeah, 50,000 lines sounds about right for 1m tokens.

If your codebase is larger than that there are a few tricks.

The first is to be selective about what you feed into the LLM: if you know the work you are doing is in a particular area of the codebase, just paste that bit in. The LLM can make reasonable guesses about things the code references that it can't see.

An increasingly effective trick is to arm a tool-using LLM with a tool like ripgrep (effectively the "interactively probe the project in a way that a human would" idea you suggested). Claude Code and OpenAI Codex both use this trick. The smarter models are really good at deciding what to search for and evaluating the results.

I've built tools that can run against Python code and extract just the class, function and method signatures and their docstrings - omitting the actual code. If you code is well designed and has reasonable documentation that could be enough for the LLM to understand it.

https://github.com/simonw/symbex is my CLI tool for that

https://simonwillison.net/2025/Apr/23/llm-fragment-symbex/ is a tool I released this morning that turns Symbex into a plugin for my LLM tool.

I use my https://llm.datasette.io/ tool a lot, especially with its new fragments feature: https://simonwillison.net/2025/Apr/7/long-context-llm/

This means I can feed in the exact code that the model needs in order to solve a problem. Here's a recent example:

  llm -m openai/o3 \
    -f https://raw.githubusercontent.com/simonw/llm-hacker-news/refs/heads/main/llm_hacker_news.py \
    -f https://raw.githubusercontent.com/simonw/tools/refs/heads/main/github-issue-to-markdown.html \
    -s 'Write a new fragments plugin in Python that registers issue:org/repo/123 which fetches that issue
        number from the specified github repo and uses the same markdown logic as the HTML page to turn that into a fragment'

From https://simonwillison.net/2025/Apr/20/llm-fragments-github/ - I'm populating the context with the exact examples needed to solve the problem.

Earlier this year, a hackernews started quizzing me about the size and scope of the projects I worked on professionally, with the implication that I couldn't really be working on anything large or complex -- that I couldn't really be doing serious development, without using a full-fat IDE like IntelliJ. I wasn't going to dox myself or my professional work just so he could reach a conclusion he's already arrived at. The point is, to this person, beyond a certain complexity threshold -- simple command-line tools, say -- an IDE was a must, otherwise you were just leaving productivity on the table.

https://news.ycombinator.com/item?id=42511441

People are going to be making the same judgements about AI-assisted coding in the near future. Sure, you could code everything yourself for your own personal enrichment, or simply because it's fun. But that will be a pursuit for your own time. In the realm of business, it's a different story: you are either proompting, or you're effectively stealing money from your employer because you're making suboptimal use of the tools available. AI gets you to something working in production so much faster that you'd be remiss not to use it. After all, as Milt and Tim Bryce have shown, the hard work in business software is in requirements analysis and design; programming is just the last translation step.

I often think about the savage job-destroying nature of the open source community: hundreds of thousands of developers working tirelessly to unemploy as many of their peers as possible by giving away the code they've written for free.

(Interesting how people talk about AI destroying programming jobs all the time, but rarely mention the impact of billions of dollars of code being given away.)