Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling LLMs to Larger Codebases

blog.kierangill.xyz

Post reply on HN
Print view View original

101–110 of 123 posts

Re: Scaling LLMs to Larger Codebases

#101

Post by nextaccountic »

jarjoura wrote: post #59
mstank wrote: post #16

As the models have progressively improved (able to handle more complex code bases, longer files, etc) I’ve started using this simple framework on repeat which seems to work pretty well at one shorting complex fixes or new features. [Research] ask the agent to explain current functionality as a way to load the right files into context. [Plan] ask the agent to brainstorm the best practices way to implement a new featur…

As of Dec 2025, Sonnet/Opus and GPTCodex are both trained and most good agent tools (ie. opencode, claude-code, codex) have prompts to fire off subagents during an exploration (use the word explore) and you should be able to Research without needing the extra steps of writing plans and resetting context. I'd save that expense unless you need some huge multi-step verifiable plan implemented. The biggest gotcha I found…

> As of Dec 2025, Sonnet/Opus and GPTCodex are both trained and most good agent tools (ie. opencode, claude-code, codex) have prompts to fire off subagents during an exploration (use the word explore) and you should be able to Research without needing the extra steps of writing plans and resetting context. I'd save that expense unless you need some huge multi-step verifiable plan implemented.

Does the UI shows clearly what portion was done by a subagent?

Re: Scaling LLMs to Larger Codebases

#102

Post by Ayanonymous »

I'm still learning about how LLMs can be used in coding, but this article helped me understand the importance of giving clear instructions and not relying too much on automation. The point about developers still needing to guide the model really makes sense. Thanks for sharing this!

Re: Scaling LLMs to Larger Codebases

#104

Post by saxenaabhi »

avereveard wrote: post #58
alienbaby wrote: post #47

Earlier quoted context omitted.

I'm curious in what kinda if situations you are seeing the model the do opposite of your intention consistently where the instructions were not complex. Do you have any examples?

Mostly gemini 3 pro when I ask to investigate a bug and provide fixing options (i do this mostly so i can see when the model loaded the right context for large tasks) gemini immediately starts fixing things and I just cant trust it Codex and claude give a nice report and if I see they're not considering this or that I can tell em.

fyi that happened to me with codex.

but, why is it a big issue? if it does something bad, just reset the worktree and try again with a different model/agent? They are dirt cheap at 20/m and I have 4 subscription(claude, codex, cursor, zed).

Re: Scaling LLMs to Larger Codebases

#105

Post by marcosdumay »

dmoy wrote: post #60
lnx01 wrote: post #46

LLMs are so good at telling me about things I know little to nothing about, but when when I ask about things I have expert knowledge on they consistently fail, hallucinate, and confidently lie...

Feels like https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

It was a very clear sarcastic reference to it.

But it's still not completely right. LLMs are actually great to tell you about things you know little about. You just have to take names, ideas, and references from it, not facts.

(And that makes agentic coding almost useless, by the way.)

Re: Scaling LLMs to Larger Codebases

#106

Post by PeterFBell »

ewoodrich wrote: post #82
mstank wrote: post #53

Earlier quoted context omitted.

In my experience this was an issue 6-8 months ago. Ever since Sonnet 4 I haven’t had any issues with instruction following. Biggest step-change has been being able to one-shot file refactors (using the planning framework I mentioned above). 6 months ago refactoring was a very delicate dance and now it feels like it’s pretty much streamlined.

I recently ran into two baffling, what felt like GPT 3.5 era completely backwards misinterpretations of an unambiguous sentence once each in Codex and CC/Sonnet a few days apart in completely different scenarios (both very early in the context window). And to be fair, they were notable partially as an "exception that proves the rule" where it was surprising to see but OP's example can definitely still happen in my ex…

It's complicated. Firstly, don't love that this happens. But the fact you're not willing to provide tolerance to a commercial tool that costs maybe a few hundred bucks a month but are willing to do so for a human who probably costs thousands of bucks a month is revealing of a double standard we're all navigating.

Its like the fallout when a waymo kills a "beloved neighborhood cat". I'm not against cats, and I'm deeply saddened at the loss of any life, but if it's true that (comparable) mile for mile, waymos reduce deaths and injuries, that is a good thing - even if they don't reduce them to zero.

And to be clear, I often feel the same way - but I am wondering why and whether it's appropriate!

Re: Scaling LLMs to Larger Codebases

#107

Post by master_crab »

nextaccountic wrote: post #101
jarjoura wrote: post #59

Earlier quoted context omitted.

As of Dec 2025, Sonnet/Opus and GPTCodex are both trained and most good agent tools (ie. opencode, claude-code, codex) have prompts to fire off subagents during an exploration (use the word explore) and you should be able to Research without needing the extra steps of writing plans and resetting context. I'd save that expense unless you need some huge multi-step verifiable plan implemented. The biggest gotcha I found…

> As of Dec 2025, Sonnet/Opus and GPTCodex are both trained and most good agent tools (ie. opencode, claude-code, codex) have prompts to fire off subagents during an exploration (use the word explore) and you should be able to Research without needing the extra steps of writing plans and resetting context. I'd save that expense unless you need some huge multi-step verifiable plan implemented. Does the UI shows clearl…

The UI (terminal) in Claude code will tell you if it has launched a subagent to research a particular file or problem. But it will not be highlighted for you, simply displayed in its record of prompts and actions.

Re: Scaling LLMs to Larger Codebases

#108

Post by the_sleaze_ »

boredtofears wrote: post #85
the_sleaze_ wrote: post #79

Earlier quoted context omitted.

I know this is tongue in cheek, but writing functional code in an object oriented language, or even worse just taking a giant procedural trail of tears and spreading it across a few files like a roomba through a pile of dog doo is ... well.. a code smell at best. I have a user prompt saved called clean code to make a pass through the changes and remove unused, DRY and refactor - literally the high points of uncle bob…

Care to share the prompt? Sounds useful!

Sure. Please improve it and come back around to let me know.

https://gist.github.com/prostko/5cf33aba05680b722017fdc0937f...

Re: Scaling LLMs to Larger Codebases

#109

Post by prmph »

saxenaabhi wrote: post #104
avereveard wrote: post #58

Earlier quoted context omitted.

Mostly gemini 3 pro when I ask to investigate a bug and provide fixing options (i do this mostly so i can see when the model loaded the right context for large tasks) gemini immediately starts fixing things and I just cant trust it Codex and claude give a nice report and if I see they're not considering this or that I can tell em.

fyi that happened to me with codex. but, why is it a big issue? if it does something bad, just reset the worktree and try again with a different model/agent? They are dirt cheap at 20/m and I have 4 subscription(claude, codex, cursor, zed).

The issue is that if it's struggling sometimes with basic instruction following, it's likely to be making insidious mistakes in large complex tasks that you might no have the wherewithal or time to review.

The thing about good abstractions is that you should be able to trust in a composable way. The simpler or more low-level the building blocks, the more reliable you should expect them to be. In LLMs you can't really make this assumption.

Re: Scaling LLMs to Larger Codebases

#110

Post by prmph »

PeterFBell wrote: post #106
ewoodrich wrote: post #82

Earlier quoted context omitted.

I recently ran into two baffling, what felt like GPT 3.5 era completely backwards misinterpretations of an unambiguous sentence once each in Codex and CC/Sonnet a few days apart in completely different scenarios (both very early in the context window). And to be fair, they were notable partially as an "exception that proves the rule" where it was surprising to see but OP's example can definitely still happen in my ex…

It's complicated. Firstly, don't love that this happens. But the fact you're not willing to provide tolerance to a commercial tool that costs maybe a few hundred bucks a month but are willing to do so for a human who probably costs thousands of bucks a month is revealing of a double standard we're all navigating. Its like the fallout when a waymo kills a "beloved neighborhood cat". I'm not against cats, and I'm deepl…

For me I was just pointing out some interesting and noteworthy failure modes.

And it matters. If the models struggle sometimes with basic instruction following, they're can quite possibly make insidious mistakes in large complex tasks that you might no have the wherewithal or time to review.

The thing about good abstractions is that you should be able to trust in a composable way. The simpler or more low-level the building blocks, the more reliable you should expect them to be. In LLMs you can't really make this assumption.

Post reply on HN