I expect Pi is mostly used with OpenAI plans, and OpenAI has a dedicated compaction endpoint you should probably be using with their models instead of a compaction prompt.
How Compaction Works in Pi
11–20 of 100 posts
Re: How Compaction Works in Pi
#12I don't like any of current solutions when it comes to compaction. I'd love to have a way to say what exactly should be summarized, because most of the time I just need to compact some noisy MCP tool calls, test runs and things like that. Just let me pick what should be summarized and keep the rest as is.
Re: How Compaction Works in Pi
#13I don't like any of current solutions when it comes to compaction. I'd love to have a way to say what exactly should be summarized, because most of the time I just need to compact some noisy MCP tool calls, test runs and things like that. Just let me pick what should be summarized and keep the rest as is.
You can do that in Pi!
> Extensions can intercept and customize both compaction and branch summarization
https://pi.dev/docs/latest/compaction
Just make an extension (or ask Pi to write an extension for itself) that intercepts compaction and leaves only what you want, or rewrites it in any other way. Should be just a few lines.
Re: How Compaction Works in Pi
#14Compaction has been a pretty painful part of local llm usage. Scrapping the current context and parsing almosy 128k of context then generating something like 5-10k tokens - that can take quite a while when you’re working with 10t/s-45t/s (depending on the model). I pretty much just start a new session whenever i fill the context.
In my opinion this is one of the areas where GPUs provide a qualitatively different experience than unified memory boxes.
For an EPYC with a 5090 (no layers on CPU) vs an M3 max 128GB, qwen 3.6 27B at 128k context / 7k generation:
Cold: prefill + decode Hot (KV cached)
5090 40s + 2-3m = 3-4 min 2-3 min
M3 Max 128GB 14m + 8-10m = 22-25 min 8-10 min
Re: How Compaction Works in Pi
#15Was expecting the article to go more in-depth.
Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
Re: How Compaction Works in Pi
#16[flagged]
Re: How Compaction Works in Pi
#17Was expecting the article to go more in-depth. Say, what happens when chain of summaries grows so long, that it still overflows context window. Is summarization runned over the summaries in the context window?
What more depth is there to go to? Compaction is a single LLM call (practically) which can have some deterministic diffing/extraction baked in, or multiple LLM calls (generally wasteful). There's only 1 summary in the context window at one time. Every prompt goes [CONVERSATION_HISTORY] + input -> model turn. As soon as total context exceeds that it compacts, so there's no summarisation overflow (you can enable an agent to access past summarisations from past compactions, but the snake starts to eat it's own tail).
Re: How Compaction Works in Pi
#18[flagged]
Yeah, and where's Bombadil Inc.?
Re: How Compaction Works in Pi
#19I don't like any of current solutions when it comes to compaction. I'd love to have a way to say what exactly should be summarized, because most of the time I just need to compact some noisy MCP tool calls, test runs and things like that. Just let me pick what should be summarized and keep the rest as is.
Sounds like you might like subagents. Agent > subagent receives agent context (presumably cached)->tool call->compact/summarise->return to main agent
Re: How Compaction Works in Pi
#20TLDR: It keeps ~20k tokens of recent conversations, then hands the rest of the conversation to another model with a special system & user prompt. This then fills out a template with relevant information. See: https://github.com/earendil-works/pi/blob/main/packages/codi...
Sounds just like opencode.