31–40 of 93 posts
At Zenning AI, a generalist AI designed to replace entire jobs with just prompts. Our agents typically run autonomously for hours, so effective context management is critical. I'd say that we invest most of our engineering effort into what is ultimately context management, such as: 1. Multi-agent orchestration 2. Summarising and chunking large tool and agent responses 3. Passing large context objects by reference bet…
Context editing is interesting because most agents work on the assumption that KV cache is the most important thing to optimise and are very hesitant to remove parts of the context during work. It also sometimes introduces hallucinations, because parts of the context are with the assumption that eg tool results are there, but theyre not. Example Manus [0]. Eg, read file A, make changes on A. Then prompt on some more…
Context editing: Replace tool call results in message history (i.e replace a file output with an indicator that it’s no longer available).
Memory: Give LLM access to read and write .md files like a virtual file system
I feel like these formalizations of tools are on the path towards managing message history on the server, which means better vendor lock in, but not necessarily a big boon to the user of the API (well, bandwidth and latency will improve). I see the ChatGPT Responses API going a similar path, and together these changes will make it harder to swap transparently between providers, something I enjoy having the ability to do.
Context editing is interesting because most agents work on the assumption that KV cache is the most important thing to optimise and are very hesitant to remove parts of the context during work. It also sometimes introduces hallucinations, because parts of the context are with the assumption that eg tool results are there, but theyre not. Example Manus [0]. Eg, read file A, make changes on A. Then prompt on some more…
so this is what claude code 2 uses under the hood? at least i got the impression it stays much better on track than the old version
Let's say I submit or let it create a piece of code, and we're working on improving it. At some point I want to consider the piece of code to be significantly better that what I had initially, so all those initial interactions containing old code could be removed from the context.
I like how Google AI Studio allows one to delete sections and they are then no longer part of the context. Not possible in Claude, ChatGPT or Gemini, I think there one can only delete the last response.
Maybe even AI could suggest which parts to disable.
I wish every instruction and response had a enable/disable checkbox so that I can disable parts of the conversation in such a way that it is excluded from the context. Let's say I submit or let it create a piece of code, and we're working on improving it. At some point I want to consider the piece of code to be significantly better that what I had initially, so all those initial interactions containing old code could…
Funny, I was just talking about my personal use of these techniques recently (tool output summarization/abliteration with memory backend). This isn't something that needs to be Claude Code specific though, you can 100% implement this with tool wrappers.
I've been doing this for a bit, dropping summarized old tool output from context is a big win, but it's still level ~0 context engineering. It'll be interesting to see which of my tricks they figure out next.
Interestingly we rolled out a similar feature recently.
I am working on a World Atlas based approach to computer use agents. If the task and app environment are reused, building an atlas of states and policies might be better than observe-plan-execute. We don't rediscover from scratch how an app works every time we use it.