Context is the bottleneck for coding agents now
runnercode.com
Context is the bottleneck for coding agents now
1–10 of 193 posts
Re: Context is the bottleneck for coding agents now
#2Has anyone tried making coding agent LoRas yet, project-specific and/or framework-specific?
Re: Context is the bottleneck for coding agents now
#3IMHO, jumping from Level 2 to Level 5 is a matter of:
- Better structured codebases - we need hierarchical codebases with minimal depth, maximal orthogonality and reasonable width. Think microservices.
- Better documentation - most code documentations are not built to handle updates. We need a proper graph structure with few sources of truth that get propagated downstream. Again, some optimal sort of hierarchy is crucial here.
At this point, I really don't think that we necessarily need better agents.
Setup your codebase optimally, spin up 5-10 instances of gpt-5-codex-high for each issue/feature/refactor (pick the best according to some criteria) and your life will go smoothly
Re: Context is the bottleneck for coding agents now
#4Has anyone tried making coding agent LoRas yet, project-specific and/or framework-specific?
I know it isn’t your question exactly, and you probably know this, but the models for coding assist tools are generally fine tunes of models for coding specific purposes. Example: in OpenAI codex they use GPT-5-codex
Re: Context is the bottleneck for coding agents now
#5IMHO, jumping from Level 2 to Level 5 is a matter of: - Better structured codebases - we need hierarchical codebases with minimal depth, maximal orthogonality and reasonable width. Think microservices. - Better documentation - most code documentations are not built to handle updates. We need a proper graph structure with few sources of truth that get propagated downstream. Again, some optimal sort of hierarchy is cru…
Can you show something you have built with that workflow?
Re: Context is the bottleneck for coding agents now
#6Context is a bottleneck for humans as well. We don’t have full context when going through the code because we can’t hold full context.
We summarize context and remember summarizations of it.
Maybe we need to do this with the LLM. Chain of thought sort of does this but it’s not deliberate. The system prompt needs to mark this as a deliberate task of building summaries and notes notes of the entire code base and this summarized context of the code base with gotchas and aspects of it can be part of permanent context the same way ChatGPT remembers aspects of you.
The summaries can even be sectioned off and and have different levels of access. So if the LLM wants to drill down to a subfolder it looks at the general summary and then it looks at another summary for the sub folder. It doesn’t need to access the full summary for context.
Imagine a hierarchy of system notes and summaries. The LLM decides where to go and what code to read while having specific access to notes it left previously when going through the code. Like the code itself it never reads it all it just access sections of summaries that go along with the code. It’s sort of like code comments.
We also need to program it to change the notes every time it changes the program. And when you change the program without consulting AI, every commit you do the AI also needs to update the notes based off of your changes.
The LLM needs a system prompt that tells it to act like us and remember things like us. We do not memorize and examine full context of anything when we dive into code.
Re: Context is the bottleneck for coding agents now
#7I've noticed that chatgpt doesnt seem to be very good at understanding elapsed time. I have some long running threads and unless i prompt it with elapsed time ("it's now 7 days later") the responses act like it was 1 second after the last message.
I think this might be a good leap for agents, the ability to not just review a doc in it's current state, but to keep in context/understanding the full evolution of a document.
Re: Context is the bottleneck for coding agents now
#8Context is a bottleneck for humans as well. We don’t have full context when going through the code because we can’t hold full context. We summarize context and remember summarizations of it. Maybe we need to do this with the LLM. Chain of thought sort of does this but it’s not deliberate. The system prompt needs to mark this as a deliberate task of building summaries and notes notes of the entire code base and this s…
> remember summarizations
yes, and if you're an engineering manager you retain _out of date_ summarizations, often materially out of date.
Re: Context is the bottleneck for coding agents now
#9> Level 2 - One commit - Cursor and Claude Code work well for tasks in this size range.
I'll stop ya right there. Spending the past few weeks fixing bugs in a big multi-tier app (which is what any production software is this days). My output per bug is always one commit, often one line.
Claude is an occasional help, nothing more. Certainly not generating the commit for me!
Re: Context is the bottleneck for coding agents now
#10And they didn't see that coming ?
I gave up building agents as soon as I figured they would never scale beyond context constraint. Increase in memory and compute costs to grow the context size of these things isn't linear.