91–100 of 100 posts
Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…
Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…
check out this: https://www.morphllm.com/products/compact you can wire it into pi compaction pretty easily
In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…
In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…
I am surprised at 'removes actual tool calls/results, tool output'. Your approach with /prune seems to be 'keep the WHAT, remove the HOW (we got here)'. I would have thought that the HOW contains some useful signal.
The how is important, but I've found all of the decisions, question, answers, and results are the most important and the tool calls themselves secondary. When necessary, the tool calls can be deleted without much being lost.
In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…
This is terrible. Models have been RLed on looking at the previous tool call chain, and reasoning. No chance this does not reduce performance. The point of compaction is that it also includes useful signal from the tool outputs itself so agent does not repeat it afterwards
In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…
People nowdays have no shame and completely forget the art of plug.
Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…
OMP changed the default compaction to images ! Kinda nuts to read about. Saves the generation cost of the traditional compaction step and writes the context as tiny text to an image, if I was following correctly.
But what if the model you're using doesn't have image processing capabilities?
Was expecting the article to go more in-depth. Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
What more depth is there to go to? Compaction is a single LLM call (practically) which can have some deterministic diffing/extraction baked in, or multiple LLM calls (generally wasteful). There's only 1 summary in the context window at one time. Every prompt goes [CONVERSATION_HISTORY] + input -> model turn. As soon as total context exceeds that it compacts, so there's no summarisation overflow (you can enable an age…
1. What happens if it overflows during assistant's turn, while it makes tool calls? Is it better to make a compaction in the middle of the chain of tool calls, or maybe before a potentially long chain of tool calls? If latter, how to choose the right point for compaction?
2. There should be enough space in the context window for a summary. What if, theoretically, a single new user message already overflows the context window? Or what happens if a summary is too long?
Maybe those are stupid questions, but I'm making a point that there is a room for going in-depth.