Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Context is the bottleneck for coding agents now

runnercode.com

Post reply on HN
Print view View original

111–120 of 193 posts

Re: Context is the bottleneck for coding agents now

#111

Post by jwpapi »

jwpapi wrote: post #105

I’m really wondering why so many advertising posts mimicked as discourse make it to frontpage and I assume it’s a new Silicon Valley trick because there is no way HN community values these so much. Let me tell you I’m scared of these tools. With Aider I have the most human in the loop possible each AI action is easy to undo, readable and manageable. However even here most of the time I want AI to write a bulk of code…

Here I think that the problem with the context is in the mind of business and dev not everything is written down and even if I would be translating it understandable (prompting) will sometimes be more work than to build it on the go with modern idea and typesafe languages

Re: Context is the bottleneck for coding agents now

#112

Post by Someone1234 »

AnotherGoodName wrote: post #73
Someone1234 wrote: post #42

Earlier quoted context omitted.

Yeah, I have the same issue too. Even for a file with several thousand lines, they will "forget" earlier parts of the file they're still working in resulting in mistakes. They don't need full awareness of the context, but they need a summary of it so that they can go back and review relevant sections. I have multiple things I'd love LLMs to attempt to do, but the context window is stopping me.

I do take that as a sign to refactor when it happens though. Even if not for the sake of LLM compatibility with the codebase it cuts down merge conflicts to refactor large files. In fact I've found LLMs are reasonable at the simple task of refactoring a large file into smaller components with documentation on what each portion does even if they can't get the full context immediately. Doing this then helps the LLM lat…

I'm trying to use LLMs to save me time and resources, "refactor your entire codebase, so the tool can work" is the opposite of that. Regardless of how you rationalize it.

Re: Context is the bottleneck for coding agents now

#113

Post by add-sub-mul-div »

jwpapi wrote: post #105

I’m really wondering why so many advertising posts mimicked as discourse make it to frontpage and I assume it’s a new Silicon Valley trick because there is no way HN community values these so much. Let me tell you I’m scared of these tools. With Aider I have the most human in the loop possible each AI action is easy to undo, readable and manageable. However even here most of the time I want AI to write a bulk of code…

You're on a site that exists to advertise job postings from YC companies, and does not stop people from spamming their personal or professional projects/companies, even when they have no activity here other than self promotion. This is an advertising site.

Re: Context is the bottleneck for coding agents now

#114

Post by daxfohl »

perplex wrote: post #35
bhu8 wrote: post #3

IMHO, jumping from Level 2 to Level 5 is a matter of: - Better structured codebases - we need hierarchical codebases with minimal depth, maximal orthogonality and reasonable width. Think microservices. - Better documentation - most code documentations are not built to handle updates. We need a proper graph structure with few sources of truth that get propagated downstream. Again, some optimal sort of hierarchy is cru…

I've been using claude on two codebases, one with good layering and clean examples, the other not so much. I get better output from the LLM with good context and clean examples and documentation. Not surprising that clarity in code benefits both humans and machines.

I think there will be a couple benefits of using agents soon. Should result in a more consistent codebase, which will make patterns easier to see and work with, and also less reinventing the wheel. Also migrations should be way faster both within and across teams, so a lot less struggling with maintaining two ways of doing something for years, which again leads to simpler and more consistent code. Finally the increased speed should lead to more serializability of feature additions, so fewer problems trying to coordinate changes happening in parallel, conflicts, redundancies, etc.

I imagine over time we'll restructure the way we work to take advantage of these opportunities and get a self-reinforcing productivity boost that makes things much simpler, though agents aren't quite capable enough for that breakthrough yet.

Re: Context is the bottleneck for coding agents now

#115

Post by keeda »

dingnuts wrote: post #55
bgirard wrote: post #31

Earlier quoted context omitted.

I think that's the real issue. If the LLM spends a lot of context investigating a bad solution and you redirect it, I notice it has trouble ignoring maybe 10K tokens of bad exploration context against my 10 line of 'No, don't do X, explore Y' instead.

that's because a next token predictor can't "forget" context. That's just not how it works. You load the thing up with relevant context and pray that it guides the generation path to the part of the model that represents the information you want and pray that the path of tokens through the model outputs what you want That's why they have a tendency to go ahead and do things you tell them not to do.. also IDK about yo…

> I didn't get into this career to be a fucking tech priest for the machine god.

You may appreciate this illustration I made (largely with AI, of course): https://imgur.com/a/0QV5mkS

The context (heheheh) is a long-ass article on coding with AI I wrote eons ago that nobody ever read, if anybody is curious: https://news.ycombinator.com/item?id=40443374

Looking back at it, I was off on a few predictions but a number of them are coming true.

Re: Context is the bottleneck for coding agents now

#116

Post by lomase »

ninetyninenine wrote: post #32
lomase wrote: post #25

Earlier quoted context omitted.

I mean... have you ever heard of this small tool called GIT that people use to track code changes?

I’m not talking about git diffs. I’m talking about the summaries of context. Every commit the ai needs to update the summaries and notes it took about the code. Did you read the entirety of what I wrote? Please read. Say the AI left a 5 line summary of a 300 line piece of code. You as a human update that code. What I am saying specifically is this: when you do the change, The AI then sees this and updates the summary…

The context is the code I work on because I can read and understand it.

If I need more, there is git, tickets, I can ask the person who wrote the code.

I do have read your comment, don't make snarky comments.

Re: Context is the bottleneck for coding agents now

#117

Post by throwacct »

We stopped hiring a while ago because we were adjusting to "AI". We're planning to start hiring next year, as upper management finally saw the writing on the wall: LLMs won't evolve past junior engineers, and we need to train junior engineers to become mid-level and senior engineers to keep the engine moving.

We're now using LLMs as mere tools (which is what it was meant to be from the get-go) to help us with different tasks, etc., but not to replace us, since they understand you need experienced and knowledgeable people to know what they're doing, since they won't learn everything there's to know to manage, improve and maintain tech used in our products and services. That sentiment will be the same for doctors, lawyers, etc., and personally, I won't put my life in the hands of any LLMs when it comes to finances, health, or personal well-being, for that matter.

If we get AGI, or the more sci-fi one, ASI, then all things will radically change (I'm thinking humanity reaching ASI will be akin to the episode from Love, Death & Robots: "When the Yogurt Took Over"). In the meantime, the hype cycle continues...

Re: Context is the bottleneck for coding agents now

#118

Post by tra3 »

tptacek wrote: post #107
tra3 wrote: post #104

Earlier quoted context omitted.

I keep wondering if we're forgetting the fundamentals: > Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it? https://www.laws-of-software.com/laws/kernighan/ Sure, you eat the elephant one bite at a time, and recursion is a thing but I wonder where the tipping point here is.

I think recursion is the wrong way to look at this, for what it's worth.

Recursion and memoization only as a general approach to solving "large" problems.

I really want to paraphrase kernighan's law as applied to LLMs. "If you use your whole context window to code a solution to a problem, how are you going to debug it?".

Post reply on HN