Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling LLMs to Larger Codebases

blog.kierangill.xyz

Post reply on HN
Print view View original

51–60 of 123 posts

Re: Scaling LLMs to Larger Codebases

#51

Post by CPLX »

smallerize wrote: post #9

This highlights a missing feature of LLM tooling, which is asking questions of the user. I've been experimenting with Gemini in VS Code, and it just fills in missing information by guessing and then runs off writing paragraphs of design and a bunch of code changes that could have been avoided by asking for clarification at the beginning.

You'd have to make it do that. Here's a cut and paste I keep open on my desktop, I just paste it back in every time things seem to drift:

> Before you proceed, read the local and global Claude.md files and make sure you understand how we work together. Make sure you never proceed beyond your own understanding.

> Always consult the user anytime you reach a judgment call rather than just proceeding. Anytime you encounter unexpected behavior or errors, always pause and consider the situation. Rather than going in circles, ask the user for help; they are always there and available.

> And always work from understanding; never make assumptions or guess. Never come up with field names, method names, or framework ideas without just going and doing the research. Always look at the code first, search online for documentation, and find the answer to things. Never skip that step and guess when you do not know the answer for certain.

And then the Claude.md file has a much more clearly written out explanation of how we work together and how it's a consultative process where every major judgment call should be prompted to the user, and every single completed task should be tested and also asked for user confirmation that it's doing what it's supposed to do. It tends to work pretty well so far.

Re: Scaling LLMs to Larger Codebases

#52

Post by vivin »

ericmcer wrote: post #37
vivin wrote: post #11

You can't get away from the engineering part of software engineering even if you are using LLMs. I have been using Claude Opus 4.5, and it's the best out of the models I have tried. I find that I can get Claude to work well if I already know the steps I need to do beforehand, and I can get it to do all of the boring stuff. So it's a series of very focused and directed one-shot prompts that it largely gets correct, be…

seriously, I stopped agent mode altogether. I hit it with very specific like: write a function that takes an array of X and returns y. It almost never fails and usually does it in a neat way, plus its ~50 lines of code so I can copy and paste confidently. Letting the agent just go wild on my code has always been a PITA for me.

I've used agent mode, but I tell it not to go hog wild and to not do anything other than what I have instructed it to do. Also, sometimes I will tell it not to change the code, and to go over its changes with me first, before I tell it that it can make the changes.

I feel the same way as you in general -- I don't trust it to go and just make changes all over the codebase. I've seen it do some really dumb stuff before because it doesn't really understand the context properly.

Re: Scaling LLMs to Larger Codebases

#53

Post by mstank »

prmph wrote: post #30
mstank wrote: post #16

As the models have progressively improved (able to handle more complex code bases, longer files, etc) I’ve started using this simple framework on repeat which seems to work pretty well at one shorting complex fixes or new features. [Research] ask the agent to explain current functionality as a way to load the right files into context. [Plan] ask the agent to brainstorm the best practices way to implement a new featur…

Nothing will really work when the models fail at the most basic of reasoning challenges. I've had models do the complete opposite of what I've put in the plan and guidelines. I've had them go re-read the exact sentences, and still see them come to the opposite conclusion, and my instructions are nothing complex at all. I used to think one could build a workflow and process around LLMs that extract good value from the…

In my experience this was an issue 6-8 months ago. Ever since Sonnet 4 I haven’t had any issues with instruction following.

Biggest step-change has been being able to one-shot file refactors (using the planning framework I mentioned above). 6 months ago refactoring was a very delicate dance and now it feels like it’s pretty much streamlined.

Re: Scaling LLMs to Larger Codebases

#54

Post by mstank »

redrove wrote: post #27
AlexB138 wrote: post #20

Earlier quoted context omitted.

This is essentially my exact workflow. I also keep the plan markdown files around in the repo to refer agents back to when adding new features. I have found it to be a really effective loop, and a great way to reprime context when returning to features.

I use an Obsidian MCP to essentially keep a database of plans, or versions sometimes that I can just fire off.

Why eat up the context with an MCP when a ./docs/plans folder does the same?

Re: Scaling LLMs to Larger Codebases

#55

Post by doublerebel »

mstank wrote: post #22
AlexB138 wrote: post #20

Earlier quoted context omitted.

This is essentially my exact workflow. I also keep the plan markdown files around in the repo to refer agents back to when adding new features. I have found it to be a really effective loop, and a great way to reprime context when returning to features.

Exactly this. I clear the old plans every few weeks. For really big features or plans I’ll ask the agent to create linear issue tickets to track progress for each phase over multiple sessions. Only MCP I have loaded is usually linear but looking for a good way to transition it to a skill.

I like Linearis as a CLI/skill interface to Linear, its help and json output are built well for use with Agents.

Re: Scaling LLMs to Larger Codebases

#57

Post by JamesSwift »

mstank wrote: post #22
AlexB138 wrote: post #20

Earlier quoted context omitted.

This is essentially my exact workflow. I also keep the plan markdown files around in the repo to refer agents back to when adding new features. I have found it to be a really effective loop, and a great way to reprime context when returning to features.

Exactly this. I clear the old plans every few weeks. For really big features or plans I’ll ask the agent to create linear issue tickets to track progress for each phase over multiple sessions. Only MCP I have loaded is usually linear but looking for a good way to transition it to a skill.

In general anything with an API is simply saying "find the auth token at ~/.config/foo.json". It mostly knows the rest endpoints and can figure out the rest

Re: Scaling LLMs to Larger Codebases

#58

Post by avereveard »

alienbaby wrote: post #47
prmph wrote: post #30

Earlier quoted context omitted.

Nothing will really work when the models fail at the most basic of reasoning challenges. I've had models do the complete opposite of what I've put in the plan and guidelines. I've had them go re-read the exact sentences, and still see them come to the opposite conclusion, and my instructions are nothing complex at all. I used to think one could build a workflow and process around LLMs that extract good value from the…

I'm curious in what kinda if situations you are seeing the model the do opposite of your intention consistently where the instructions were not complex. Do you have any examples?

Mostly gemini 3 pro when I ask to investigate a bug and provide fixing options (i do this mostly so i can see when the model loaded the right context for large tasks) gemini immediately starts fixing things and I just cant trust it

Codex and claude give a nice report and if I see they're not considering this or that I can tell em.

Re: Scaling LLMs to Larger Codebases

#59

Post by jarjoura »

mstank wrote: post #16

As the models have progressively improved (able to handle more complex code bases, longer files, etc) I’ve started using this simple framework on repeat which seems to work pretty well at one shorting complex fixes or new features. [Research] ask the agent to explain current functionality as a way to load the right files into context. [Plan] ask the agent to brainstorm the best practices way to implement a new featur…

As of Dec 2025, Sonnet/Opus and GPTCodex are both trained and most good agent tools (ie. opencode, claude-code, codex) have prompts to fire off subagents during an exploration (use the word explore) and you should be able to Research without needing the extra steps of writing plans and resetting context. I'd save that expense unless you need some huge multi-step verifiable plan implemented.

The biggest gotcha I found is that these LLMs love to assume that code is C/Python but just in your favorite language of choice. Instead of considering that something should be written encapsulated into an object to maintain state, it will instead write 5 functions, passing the state as parameters between each function. It will also consistently ignore most of the code around it, even if it could benefit from reading it to know what specifically could be reused. So you end up with copy-pasta code, and unstructured copy-pasta at best.

The other gotcha is that claude usually ignores CLAUDE.md. So for me, I first prompt it to read it and then I prompt it to next explore. Then, with those two rules, it usually does a good job following my request to fix, or add a new feature, or whatever, all within a single context. These recent agents do a much better job of throwing away useless context.

I do think the older models and agents get better results when writing things to a plan document, but I've noticed recent opus and sonnet usually end up just writing the same code to the plan document anyway. That usually ends up confusing itself because it can't connect it to the code around the changes as easily.

Post reply on HN