Scaling LLMs to Larger Codebases
blog.kierangill.xyz
Scaling LLMs to Larger Codebases
1–10 of 123 posts
Re: Scaling LLMs to Larger Codebases
#2Or why you shouldn't....
Re: Scaling LLMs to Larger Codebases
#3STAN'd to the top.
Decent article but it feels like a linkedin rehashing of stuff the people at the edge have already known for a while.
Re: Scaling LLMs to Larger Codebases
#4> Making a prompt library useful requires iteration. Every time the LLM is slightly off target, ask yourself, "What could've been clarified?" Then, add that answer back into the prompt library.
I'm far from an LLM power user, but this is the single highest ROI practice I've been using.
You have to actually observe what the LLM is trying to do each time. Simply smashing enter over and over again or setting it to auto-accept everything will just burn tokens. Instead, see where it gets stuck and add a short note to CLAUDE.md or equivalent. Break it out into sub-files to open for different types of work if the context file gets large.
Letting the LLM churn and experiment for every single task will make your token quota evaporate before your eyes. Updating the context file constantly is some extra work for you, but it pays off.
My primary use case for LLMs is exploring code bases and giving me summaries of which files to open, tracing execution paths through functions, and handing me the info I need. It also helps a lot to add some instructions for how to deliver useful results for specific types of questions.
Re: Scaling LLMs to Larger Codebases
#5STAN'd to the top. Decent article but it feels like a linkedin rehashing of stuff the people at the edge have already known for a while.
> but it feels like a linkedin rehashing of stuff the people at the edge have already known for a while.
You're not wrong, but it bears repeating to newcomers.
The average LLM user I encounter is still just hammering questions into the prompt and getting frustrated when the LLM makes the same mistakes over and over again.
Re: Scaling LLMs to Larger Codebases
#6Its kind of crazy that the knee jerk reaction to failing to one shot your prompt is to abandon the whole thing because you think the tool sucks. It very well might, but it could also be user error or a number of other things. There wouldn't be a good nights sleep in sight if I knew an LLM was running rampant all over production code in an effort to "scale it".
Re: Scaling LLMs to Larger Codebases
#7> Making a prompt library useful requires iteration. Every time the LLM is slightly off target, ask yourself, "What could've been clarified?" Then, add that answer back into the prompt library. I'm far from an LLM power user, but this is the single highest ROI practice I've been using. You have to actually observe what the LLM is trying to do each time. Simply smashing enter over and over again or setting it to auto-…
I'm with you on that, but I have to say I have been doing that aggressively, and it's pretty easy for Claude Code at least to ignore the prompts, commands, Markdown files, README, architecture docs, etc.
I feel like I spend quite a bit of time telling the thing to look at information it already knows. And I'm talking about when I HAVE actually created various documents to use and prompts.
As a specific example, it regularly just doesn't reference CLAUDE.md and it seems pretty random as to when it decides to drop that out of context. That's including right at session start when it should have it fresh.
Re: Scaling LLMs to Larger Codebases
#8> Making a prompt library useful requires iteration. Every time the LLM is slightly off target, ask yourself, "What could've been clarified?" Then, add that answer back into the prompt library. I'm far from an LLM power user, but this is the single highest ROI practice I've been using. You have to actually observe what the LLM is trying to do each time. Simply smashing enter over and over again or setting it to auto-…
I'm with you on that, but I have to say I have been doing that aggressively, and it's pretty easy for Claude Code at least to ignore the prompts, commands, Markdown files, README, architecture docs, etc. I feel like I spend quite a bit of time telling the thing to look at information it already knows. And I'm talking about when I HAVE actually created various documents to use and prompts. As a specific example, it re…
> and it's pretty easy for Claude Code at least to ignore the prompts, commands, Markdown files, README, architecture docs, etc.
I would agree with that!
I've been experimenting with having Claude re-write those documents itself. It can take simple directives and turn them into hierarchical Markdown lists that have multiple bullet points. It's annoying and overly verbose for humans to read, but the repetition and structure seems to help the LLM.
I also interrupt it and tell it to refer back to CLAUDE.md if it gets too off track.
Like I said, though, I'm not really an LLM power user. I'd be interested to hear tips from others with more time on these tools.
Re: Scaling LLMs to Larger Codebases
#9This highlights a missing feature of LLM tooling, which is asking questions of the user. I've been experimenting with Gemini in VS Code, and it just fills in missing information by guessing and then runs off writing paragraphs of design and a bunch of code changes that could have been avoided by asking for clarification at the beginning.
Re: Scaling LLMs to Larger Codebases
#10The issues raised in this article are why I think highly-opinionated frameworks will lead to higher developer productivity when using AI assisted coding
You may not like all the opinions of the framework, but the LLM knows them and you don’t need to write up any guidelines for it.