Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Compaction Works in Pi

earendil.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: How Compaction Works in Pi

#71

Post by alfiedotwtf »

I want to like Pi but compaction is why I had to go back to Open Code…

My problem is that when in a loop and it’s calling tools, it won’t check how close it is to the compacting limit until the whole loop returns to you for the next prompt. And so if you have a run that could go for hours, it’s a gamble if you’ll OOM or an interrupting compaction breaks context and stops the loop without continuing.

There’s a few extensions that all try to solve this problem, but I’ve found none actually work :(

Re: How Compaction Works in Pi

#72

Post by marton78 »

kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

[dead]

Re: How Compaction Works in Pi

#73

Post by marton78 »

WhyNotHugo wrote: post #54
kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

Potentially: remove thinking blocks, and keep the rest. At least this would ensure that the entire context of the conversation is still there, and anything said isn't lost. Having a second model also iterate the resulting messages and remove low-value tool calls could also be interesting. Especially failed calls which add no value.

context-fold does this:

https://pi.dev/packages/context-fold

Re: How Compaction Works in Pi

#74

Post by badlogic »

navs wrote: post #51

Ampcode used a handoff feature for a while that I found genuinely useful [1] and then they removed it. Anecdotally, I felt it worked better than compaction. [1] https://ampcode.com/news/handoff

pi has /tree, which is basically the same, except in the same session.

Re: How Compaction Works in Pi

#75

Post by Schlagbohrer »

mikepurvis wrote: post #38
kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

I'm very interested in this too. I feel like when my Claude session compacts I immediately have to re-alert it to critical aspects of the task, but it should be possible even to have a secondary low-skill agent crawl over the whole context window semi-continuously and nominate pieces for removal or summarization. Or at a higher level, have something able to decompose, like hey I realised we're actually working on thr…

I have run in to the same thing. Most recently I told my pi-agent to fork a git repo since our work had morphed into two different tasks that shouldn't be packed into the same context and workflow. Git fork is what I will use as a tool for this division in the future.

Re: How Compaction Works in Pi

#76

Post by bvrmn »

errantmind wrote: post #50

In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…

People nowdays have no shame and completely forget the art of plug.

Re: How Compaction Works in Pi

#77

Post by imgyuri »

the way compaction works is pretty simple and highly relies on another model to do it, the only part the user has control of is when to do the compaction, which actually means when doing large amount of work in one shot, it should be planned from the beginning to be separated into works that can be reviewed, afterwards the model can compact

Re: How Compaction Works in Pi

#78

Post by Imanari »

errantmind wrote: post #50

In my experience, the best approach to compaction is to never get to the point where you need compaction and to generally stay below about 30% context window utilization. Even for long agentic workflows this can be accomplished for quite a while, much longer than most people might think. Here's what I do for each of my sessions: 1. For asides, off-topic work, or repetitive work that has already been done in the sessi…

I am surprised at 'removes actual tool calls/results, tool output'. Your approach with /prune seems to be 'keep the WHAT, remove the HOW (we got here)'. I would have thought that the HOW contains some useful signal.

Re: How Compaction Works in Pi

#79

Post by trueno »

mikepurvis wrote: post #38
kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

I'm very interested in this too. I feel like when my Claude session compacts I immediately have to re-alert it to critical aspects of the task, but it should be possible even to have a secondary low-skill agent crawl over the whole context window semi-continuously and nominate pieces for removal or summarization. Or at a higher level, have something able to decompose, like hey I realised we're actually working on thr…

yeah ive read so many takes on how people survive compaction amnesia ive wired up my own "protocols" that are okay, have used things like beads, am okay burning a buttload of context right at the beginning of a post-compaction just to get up to speed and oriented.. but i still end up with a decent amount of sprawl and lately the writing style of claude has just gotten so dense and nonsensical that my eyes are starting to glaze over.

this seems to be the fate with all the big grand 'god' projects people are doing with AI. super epic omega knowledge bases, super epic omega personal platforms, etc. I don't think enough people admit that whatever they're working on has evolved into something they don't fully understand.

Re: How Compaction Works in Pi

#80

Post by zavec »

boorang wrote: post #52
damsta wrote: post #12

I don't like any of current solutions when it comes to compaction. I'd love to have a way to say what exactly should be summarized, because most of the time I just need to compact some noisy MCP tool calls, test runs and things like that. Just let me pick what should be summarized and keep the rest as is.

I mean, not to be flippant but can't you just prompt the agent to write a file as you're getting closer to the compaction limit? I tend to just go to roughly 50-70% context utilization and then tell the agent to summarize the conversation and save it to a file, manually /clear, then say let's continue that last conversation. You can inspect the summary first and make any changes.

The way I've done it has been when there's a longer task, I give it a markdown document that has a plan with numbered steps, and then for each step start a fresh context window and tell it to update that doc as it goes with any decisions taken or deviations from the plan, or other context needed for future steps. If it gets close to the end of the context I tell it to summarize the current state and anything a fresh context would need to know.
Post reply on HN