Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling LLMs to Larger Codebases

blog.kierangill.xyz

Post reply on HN
Print view View original

11–20 of 123 posts

Re: Scaling LLMs to Larger Codebases

#11

Post by vivin »

You can't get away from the engineering part of software engineering even if you are using LLMs. I have been using Claude Opus 4.5, and it's the best out of the models I have tried. I find that I can get Claude to work well if I already know the steps I need to do beforehand, and I can get it to do all of the boring stuff. So it's a series of very focused and directed one-shot prompts that it largely gets correct, because I'm not giving it a huge task, or something open-ended.

Knowing how you would implement the solution beforehand is a huge help, because then you can just tell the LLM to do the boring/tedious bits.

Re: Scaling LLMs to Larger Codebases

#12

Post by t_tsonev »

mym1990 wrote: post #6

Its kind of crazy that the knee jerk reaction to failing to one shot your prompt is to abandon the whole thing because you think the tool sucks. It very well might, but it could also be user error or a number of other things. There wouldn't be a good nights sleep in sight if I knew an LLM was running rampant all over production code in an effort to "scale it".

I'm okay with writing developer docs in the form of agent instructions, those are useful for humans too. If they start to get oddly specific or sound mental, then it's obviously the tool at fault.

Re: Scaling LLMs to Larger Codebases

#13

Post by zarp »

CPLX wrote: post #7
Aurornis wrote: post #4

> Making a prompt library useful requires iteration. Every time the LLM is slightly off target, ask yourself, "What could've been clarified?" Then, add that answer back into the prompt library. I'm far from an LLM power user, but this is the single highest ROI practice I've been using. You have to actually observe what the LLM is trying to do each time. Simply smashing enter over and over again or setting it to auto-…

I'm with you on that, but I have to say I have been doing that aggressively, and it's pretty easy for Claude Code at least to ignore the prompts, commands, Markdown files, README, architecture docs, etc. I feel like I spend quite a bit of time telling the thing to look at information it already knows. And I'm talking about when I HAVE actually created various documents to use and prompts. As a specific example, it re…

> it seems pretty random as to when it decides to drop that out of context

Overcoming this kind of nondeterministic behavior around creating/following/modifying instructions is the biggest thing I wish I could solve with my LLM workflows. It seems like you might be able to do this through a system of Claude Code hooks, but I've struggled with finding a good UX for maintaining a growing and ever-changing collection of hooks.

Are there any tools or harnesses that attempt to address this and allow you to "force" inject dynamic rules as context?

Re: Scaling LLMs to Larger Codebases

#15

Post by skolos »

smallerize wrote: post #9

This highlights a missing feature of LLM tooling, which is asking questions of the user. I've been experimenting with Gemini in VS Code, and it just fills in missing information by guessing and then runs off writing paragraphs of design and a bunch of code changes that could have been avoided by asking for clarification at the beginning.

Claude code regularly asks me questions - I like how anthropic implemented this

Re: Scaling LLMs to Larger Codebases

#16

Post by mstank »

As the models have progressively improved (able to handle more complex code bases, longer files, etc) I’ve started using this simple framework on repeat which seems to work pretty well at one shorting complex fixes or new features.

[Research] ask the agent to explain current functionality as a way to load the right files into context.

[Plan] ask the agent to brainstorm the best practices way to implement a new feature or refactor. Brainstorm seems to be a keyword that triggers a better questioning loop for the agent. Ask it to write a detailed implementation plan to an md file.

[clear] completely clear the context of the agent —- better results than just compacting the conversation.

[execute plan] ask the agent to review the specific plan again, sometimes it will ask additional questions which repeats the planning phase again. This loads only the plan into context and then have it implement the plan.

[review & test] clear the context again and ask it to review the plan to make sure everything was implemented. This is where I add any unit or integration tests if needed. Also run test suites, type checks, lint, etc.

With this loop I’ve often had it run for 20-30 minutes straight and end up with usable results. It’s become a game of context management and creating a solid testing feedback loop instead of trying to purely one-shot issues.

Re: Scaling LLMs to Larger Codebases

#17

Post by rockbruno »

skolos wrote: post #15
smallerize wrote: post #9

This highlights a missing feature of LLM tooling, which is asking questions of the user. I've been experimenting with Gemini in VS Code, and it just fills in missing information by guessing and then runs off writing paragraphs of design and a bunch of code changes that could have been avoided by asking for clarification at the beginning.

Claude code regularly asks me questions - I like how anthropic implemented this

Yeah I experienced this yesterday and it was really cool. It really only happened once though.

Re: Scaling LLMs to Larger Codebases

#18

Post by kierangill »

CPLX wrote: post #7
Aurornis wrote: post #4

> Making a prompt library useful requires iteration. Every time the LLM is slightly off target, ask yourself, "What could've been clarified?" Then, add that answer back into the prompt library. I'm far from an LLM power user, but this is the single highest ROI practice I've been using. You have to actually observe what the LLM is trying to do each time. Simply smashing enter over and over again or setting it to auto-…

I'm with you on that, but I have to say I have been doing that aggressively, and it's pretty easy for Claude Code at least to ignore the prompts, commands, Markdown files, README, architecture docs, etc. I feel like I spend quite a bit of time telling the thing to look at information it already knows. And I'm talking about when I HAVE actually created various documents to use and prompts. As a specific example, it re…

Agreed here. A key theme, which isn’t terribly explicit in this post, is that your codebase is your context.

I’ve found that when my agent flies off the rails, it’s due to an underlying weakness in the construction of my program. The organization of the codebase doesn’t implicitly encode the “map”. Writing a prompt library helps to overcome this weakness, but I’ve found that the most enduring guidance comes from updating the codebase itself to be more discoverable.

Re: Scaling LLMs to Larger Codebases

#19

Post by zeroonetwothree »

mym1990 wrote: post #6

Its kind of crazy that the knee jerk reaction to failing to one shot your prompt is to abandon the whole thing because you think the tool sucks. It very well might, but it could also be user error or a number of other things. There wouldn't be a good nights sleep in sight if I knew an LLM was running rampant all over production code in an effort to "scale it".

There’s always a trade off in terms of alternative approaches. So I don’t think it’s “crazy” that if one fails you switch to a different one. Sure, sometimes persistence can pay off, but not always.

Like if I go to a restaurant for the first time and the item I order is bad, could I go back and try something else? Perhaps, but I could also go somewhere else.

Re: Scaling LLMs to Larger Codebases

#20

Post by AlexB138 »

mstank wrote: post #16

As the models have progressively improved (able to handle more complex code bases, longer files, etc) I’ve started using this simple framework on repeat which seems to work pretty well at one shorting complex fixes or new features. [Research] ask the agent to explain current functionality as a way to load the right files into context. [Plan] ask the agent to brainstorm the best practices way to implement a new featur…

This is essentially my exact workflow. I also keep the plan markdown files around in the repo to refer agents back to when adding new features. I have found it to be a really effective loop, and a great way to reprime context when returning to features.
Post reply on HN