91–100 of 211 posts
I've been able to avoid context size issues by applying one simple constraint to my agent loop. What I do is prevent all tool calling in the user's top-level conversation thread. Anything that needs to tool call must happen in a recursive invoke of the agent, which returns whatever results to caller. I can keep the same high level conversation going for an entire day over a million LOC+ codebase without ever hitting…
Considerations about what goes on in agents internally will probably not be part of software development for long. Personally, I already see LLMs and agents as blackboxes. I give each feature request to multiple LLMs and then compare the results. I don't manually use "sessions" at all. I just look at the outcome. When I dislike it, I "git reset --hard", change my prompts and restart the feature request. To have an on…
Asking because I could guess that approach would be ok for the types of front end work that doesn't require much security or other validation.
But it sounds like it wouldn't be suitable for work in regulated industries or anything that needs to have extreme care taken.
?
Few of the best sessions I have ever had with claude went into 700-800k territory.
I frequently reach 400-600k without visible (to me) signs of quality regression.
This has not been my experience with Opus since Anthropic released the 1M token context window for use under the subscription plans. I routinely push past 500k tokens, even sometimes up to around 800k tokens, and don't see this problem. I've seen it to some extent when getting truly near the limit, up around and above 900k tokens, though what I see isn't as severe as the author seems to see. (And I rarely fill the co…
Earlier quoted context omitted.
Yeah 60k is ludicrous, I've barely seeded the context at that point and I don't see context related degradation until well into the 600-700k.
In this thread: People tossing coins independently and fighting over the result they got.
It seems that people have different workflows or repos, or memories or prompts or expectations.
I had the impression, models would get inconsistent after just 3000 words.
Reminds me the sign, "Do not dumb here. No dumb zone."
I've been able to avoid context size issues by applying one simple constraint to my agent loop. What I do is prevent all tool calling in the user's top-level conversation thread. Anything that needs to tool call must happen in a recursive invoke of the agent, which returns whatever results to caller. I can keep the same high level conversation going for an entire day over a million LOC+ codebase without ever hitting…
This is interesting to me because reducing context & token usage is in the user's best interest but not in the financial interest of AI vendors. I am not an expert but it sounds like your "one simple trick" would fix context issues and allow much tighter control over token usage. Thanks for being willing to share this tip in an HN comment, changing how those in the know use AI agents going forward -- it's hard to kee…
Earlier quoted context omitted.
This is interesting to me because reducing context & token usage is in the user's best interest but not in the financial interest of AI vendors. I am not an expert but it sounds like your "one simple trick" would fix context issues and allow much tighter control over token usage. Thanks for being willing to share this tip in an HN comment, changing how those in the know use AI agents going forward -- it's hard to kee…
The tokens are still being burnt, they're just doing so in a parallel dimension from the users main context window.