also, we are one prompt away from achieving AGI...
Context is the bottleneck for coding agents now
51–60 of 193 posts
Re: Context is the bottleneck for coding agents now
#52Context is a bottleneck for humans as well. We don’t have full context when going through the code because we can’t hold full context. We summarize context and remember summarizations of it. Maybe we need to do this with the LLM. Chain of thought sort of does this but it’s not deliberate. The system prompt needs to mark this as a deliberate task of building summaries and notes notes of the entire code base and this s…
youre projecting a deficiency of the human brain onto computers. computers have advantages that our brains dont (perfect and large memory), theres no reason to think that we should try to recreate how humans do things.
why would you bother with all these summaries if you can just read and remember the code perfectly.
Re: Context is the bottleneck for coding agents now
#53"And yet, coding agents are nowhere near capable of replacing software developers. Why is that?"
Because you will always need a specialist to drive these tools. You need someone who understands the landscape of software - what's possible, what's not possible, how to select and evaluate the right approach to solve a problem, how to turn messy human needs into unambiguous requirements, how to verify that the produced software actually works.
Provided software developers can grow their field of experience to cover QA and aspects of product management - and learn to effectively use this new breed of coding agents - they'll be just fine.
Re: Context is the bottleneck for coding agents now
#54Re: Context is the bottleneck for coding agents now
#55There's a misunderstanding here broadly. Context could be infinite, but the real bottleneck is understanding intent late in a multi-step operation. A human can effectively discard or disregard prior information as the narrow window of focus moves to a new task, LLMs seem incredibly bad at this. Having more context, but leaving open an inability to effectively focus on the latest task is the real problem.
I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.
that's because a next token predictor can't "forget" context. That's just not how it works.
You load the thing up with relevant context and pray that it guides the generation path to the part of the model that represents the information you want and pray that the path of tokens through the model outputs what you want
That's why they have a tendency to go ahead and do things you tell them not to do..
also IDK about you but I hate how much praying has become part of the state of the art here. I didn't get into this career to be a fucking tech priest for the machine god. I will never like these models until they are predictable, which means I will never like them.
Re: Context is the bottleneck for coding agents now
#56context and memory has been a bottleneck from like day one
Re: Context is the bottleneck for coding agents now
#57I agree, and I think intent behind the code is the most important part in missing context. You can sometimes infer intent from code, but usually code is a snapshot of an expression of an evolving intent.
Re: Context is the bottleneck for coding agents now
#58I believe if you create something like a task manager for the coding agents, think something hosted on the web like Jira, you can work around this.
I started writing a solution, but to be honest I probably need the help of someone who's more experienced.
Although to be honest, I'm sure someone with VC money is already working on this.
Re: Context is the bottleneck for coding agents now
#59There's a misunderstanding here broadly. Context could be infinite, but the real bottleneck is understanding intent late in a multi-step operation. A human can effectively discard or disregard prior information as the narrow window of focus moves to a new task, LLMs seem incredibly bad at this. Having more context, but leaving open an inability to effectively focus on the latest task is the real problem.
I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.
I think the general term for this is "context poisoning" and is related but slightly different to what the poster above you is saying. Even with a "perfect" context, the LLM still can't infer intent.
Re: Context is the bottleneck for coding agents now
#60Earlier quoted context omitted.
I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.
that's because a next token predictor can't "forget" context. That's just not how it works. You load the thing up with relevant context and pray that it guides the generation path to the part of the model that represents the information you want and pray that the path of tokens through the model outputs what you want That's why they have a tendency to go ahead and do things you tell them not to do.. also IDK about yo…
You can rewrite the history (but there are issues with that too). So an agent can forget context. Simply dont feed in part of the context on the next run.