Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Context is the bottleneck for coding agents now

runnercode.com

Post reply on HN
Print view View original

161–170 of 193 posts

Re: Context is the bottleneck for coding agents now

#161

Post by cindyllm »

ninetyninenine wrote: post #159
lomase wrote: post #116

Earlier quoted context omitted.

The context is the code I work on because I can read and understand it. If I need more, there is git, tickets, I can ask the person who wrote the code. I do have read your comment, don't make snarky comments.

So you hold all that code context in your head at the same time? > If I need more, there is git, tickets, I can ask the person who wrote the code. What does this have to do with anything? Go ahead and ask the person. The notes the LLM writes aren’t for you they are for the LLM. You do you.

[dead]

Re: Context is the bottleneck for coding agents now

#162

Post by scoopdiwhoop »

alchemist1e9 wrote: post #101
reclusive-sky wrote: post #79

Earlier quoted context omitted.

I agree, I often see Opus 4.1 and GPT5 (Thinking) make astoundingly stupid decisions with full confidence, even on trivial tasks requiring minimal context. Assuming they would make better decisions "if only they had more context" is a fallacy

Is there a good example you could provide of that? I just haven’t seen that personally so I’d be interested in any examples on these current models. I’m sure we all remember in the early days lots of examples of stupidity being posted and it was interesting. It be great if people kept doing that so we could get a better sense of which types of problems they are failing with astounding levels of stupidity on.

One example I ran into recently is asking Gemini CLI to do something that isn't possible: use multiple tokens in a Gemini CLI custom command (https://github.com/google-gemini/gemini-cli/blob/main/docs/c...). It pretended it was possible and came up with a nonsense .toml defining multiple arguments in a way it invented so it couldn't be read, even after multiple rounds of "that doesn't work, Gemini can't load this."

So in any situation where something can't actually be done my assumption is that it's just going to hallucinate a solution.

Has been good for busywork that I know how to do but want to save time on. When I'm directing it, it works well. When I'm asking it to direct me, it's gonna lead me off a cliff if I let it.

Re: Context is the bottleneck for coding agents now

#163

Post by lomase »

ninetyninenine wrote: post #159
lomase wrote: post #116

Earlier quoted context omitted.

The context is the code I work on because I can read and understand it. If I need more, there is git, tickets, I can ask the person who wrote the code. I do have read your comment, don't make snarky comments.

So you hold all that code context in your head at the same time? > If I need more, there is git, tickets, I can ask the person who wrote the code. What does this have to do with anything? Go ahead and ask the person. The notes the LLM writes aren’t for you they are for the LLM. You do you.

So you hold all that code context in your head at the same time?

Yes. That is how every single piece of code has been writen since the creation of computers.

Why you seem so surprised?

Re: Context is the bottleneck for coding agents now

#164

Post by sshine »

ericmcer wrote: post #150
bgirard wrote: post #31

Earlier quoted context omitted.

I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.

It seems possible for openAI/Anthropic to rework their tools so they discard/add relevant context on the fly, but it might have some unintended behaviors. The main thing is people have already integrated AI into their workflows so the "right" way for the LLM to work is the way people expect it to. For now I expect to start multiple fresh contexts while solving a single problem until I can setup a context that gets th…

> rework their tools so they discard/add relevant context on the fly

That may be the foundation for an innovation step in model providers. But you can achieve a poor man’s simulation if you can determine, in retrospect, when a context was at peak for taking turns, and when it got too rigid, or too many tokens were spent, and then simply replay the context up until that point.

I don’t know if evaluating when a context is worth duplicating is a thing; it’s not deterministic, and it depends on enforcing a certain workflow.

Re: Context is the bottleneck for coding agents now

#165

Post by vel0city »

ericmcer wrote: post #150
bgirard wrote: post #31

Earlier quoted context omitted.

I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.

It seems possible for openAI/Anthropic to rework their tools so they discard/add relevant context on the fly, but it might have some unintended behaviors. The main thing is people have already integrated AI into their workflows so the "right" way for the LLM to work is the way people expect it to. For now I expect to start multiple fresh contexts while solving a single problem until I can setup a context that gets th…

A number of agentic coding tools do this. Upon an initial request for a larger set of actions, it will write a markdown file with its "thoughts" on its plan to do something, and keep notes as it goes. They'll then automatically compact their contexts and re-read their notes to keep "focused" while still having a bit of insight on what it did previously and what the original ask was.

Re: Context is the bottleneck for coding agents now

#166

Post by cvzakharchenko »

vel0city wrote: post #165
ericmcer wrote: post #150

Earlier quoted context omitted.

It seems possible for openAI/Anthropic to rework their tools so they discard/add relevant context on the fly, but it might have some unintended behaviors. The main thing is people have already integrated AI into their workflows so the "right" way for the LLM to work is the way people expect it to. For now I expect to start multiple fresh contexts while solving a single problem until I can setup a context that gets th…

A number of agentic coding tools do this. Upon an initial request for a larger set of actions, it will write a markdown file with its "thoughts" on its plan to do something, and keep notes as it goes. They'll then automatically compact their contexts and re-read their notes to keep "focused" while still having a bit of insight on what it did previously and what the original ask was.

Interesting. I know people do this manually. But are there agentic coding tools that actually automate this approach?

Re: Context is the bottleneck for coding agents now

#167

Post by sshine »

cvzakharchenko wrote: post #166
vel0city wrote: post #165

Earlier quoted context omitted.

A number of agentic coding tools do this. Upon an initial request for a larger set of actions, it will write a markdown file with its "thoughts" on its plan to do something, and keep notes as it goes. They'll then automatically compact their contexts and re-read their notes to keep "focused" while still having a bit of insight on what it did previously and what the original ask was.

Interesting. I know people do this manually. But are there agentic coding tools that actually automate this approach?

Claude Code has /init and /compact that do this. It doesn’t recreate the context as-is, but creates a context that is presumed to be functionally equivalent. I find that’s not the case and that building up from very little stored context and a lot of specialised dialogue works better.

Re: Context is the bottleneck for coding agents now

#168

Post by fzzzy »

cjbgkagh wrote: post #139
PantaloonFlames wrote: post #129

Earlier quoted context omitted.

You are saying “fresh LLM” but really I think you’re referring to a curated context. The existing coding agents have mechanisms to do this. Saving context to a file. Editing the file. Clearing all context except for the file. It’s sort of clunky now but it will get better and slicker.

It seems that I have missed this existing feature, I’m only a light user of LLMs, I’ll keep an eye out for it.

some sibling comments mentioned Claude code has this

Re: Context is the bottleneck for coding agents now

#169

Post by koakuma-chan »

pu_pe wrote: post #38
neutronicus wrote: post #12

Earlier quoted context omitted.

I think the question is, can I throw a couple thousand bucks of GPU time at fine-tuning a model to have knowledge of our couple million lines of C++ baked into the weights instead of needing to fuck around with "Context Engineering". Like, how feasible is it for a mid-size corporation to use a technique like LoRA, mentioned by GP, to "teach" (say, for example) Kimi K2 about a large C++ codebase so that individual eng…

I'm curious about it too. I think there are two bottlenecks, one is that training a relatively large LLM can be resource-intensive (so people go for RAGs and other shortcuts), and making it finetuned to your use cases might make it dumber overall.

> making it finetuned to your use cases might make it dumber overall.

LoRa doesn't overwrite weights.

Re: Context is the bottleneck for coding agents now

#170

Post by koakuma-chan »

CardenB wrote: post #4
koakuma-chan wrote: post #2

Has anyone tried making coding agent LoRas yet, project-specific and/or framework-specific?

I know it isn’t your question exactly, and you probably know this, but the models for coding assist tools are generally fine tunes of models for coding specific purposes. Example: in OpenAI codex they use GPT-5-codex

I think they fine tune them for tool calling, not knowledge
Post reply on HN