Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Compaction Works in Pi

earendil.com

Post reply on HN
Print view View original

81–90 of 100 posts

Re: How Compaction Works in Pi

#81

Post by Imanari »

Great thread, I was just thinking about compaction. My current line of thought is that compaction/pruning/ctx management in general should be something ongoing and maybe recursive. For example:

User:'How is auth implemented?' -> [thinking] [codebase exploration with [thinking] in between, 10 file reads, 3 of which were "wrong"] [thinking] -> agent_response

This little exchange contains a WHAT (how auth actually is implemented) and a HOW (where that info is and how to retrieve it). Maybe this question was part of a larger task. I think that whole exchange could be summarised before it enters context, kind of like what happens with subagents. The main thread would then consist mostly of [summaries]. Eventually the context will fill up anyway and we would summarise those summaries again. Alternatively one could maintain a [master_summary], kind of like an internal state. So new [summaries] get integrated directly and the [master_summary] gets updated.

Re: How Compaction Works in Pi

#82

Post by julesrms »

In juggler (https://news.ycombinator.com/item?id=48883305) I spent a lot of effort on some compaction tricks that are elegant, but in reality find myself not really using them..

Because juggler has a nested-thread architecture, I had a great ah-ha! moment when I realised that with sub-threads, compaction becomes almost free: you just take the list of items in the conversation, move them all into a new sub-thread, then allow that sub-thread to summarise itself (this summary is what the parent thread sees). Hey-presto, your parent thread is now compacted, and the sub-thread contains all the old messages, so you can browse them if you need to, or just undo the whole operation if you change you mind. This struck me as super-elegant, and I was chuffed that it works.

However.. What I find myself actually doing now is just using the "new conversation" tool to hand over to a new conversation - I just tell the LLM "create a new conversation, and tell it everything it needs to continue this task". It gets you to the same place, but generally I'm not interested in the history, and having a fresh conversation with a new name usually feels neater.

Re: How Compaction Works in Pi

#83

Post by toasty228 »

kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

In oh-my-pi there is the /shake command that gets rids of tool results and some other things

Re: How Compaction Works in Pi

#85

Post by tyingq »

behindsight wrote: post #49
kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

OMP has this, it's called "/shake" where bloat from tool calls and such is turned into an artifact ref instead of remaining in the context. There are other pruning or compaction strategies you can configure too. You can also use "/btw" to have side tangent conversations with the current context but with the benefit that it will not bloat your current context after the tangent is done. I wish other harnesses were prio…

> You can also use "/btw" to have side tangent conversations with the current context

Ah, something I needed, but did not know I needed. Comes up ALL the time. I want to know something that is well off the task, but it's the best time to ask due to the specifically loaded context.

Re: How Compaction Works in Pi

#86

Post by infecto »

Does anyone find compaction useful? I have always been a big believer of many small chats instead of large ones. There is certainly a time and a place where maybe I need to load in a lot to initial context BUT I often find that quality goes does as we near context limits and of course costs go massively up. I would rather repeat small notarized chunks than have a massive context window.

Re: How Compaction Works in Pi

#87

Post by ivanovm »

kierangill wrote: post #22

Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages. For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration. Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations le…

check out this: https://www.morphllm.com/products/compact you can wire it into pi compaction pretty easily

Re: How Compaction Works in Pi

#88

Post by ramon156 »

infecto wrote: post #86

Does anyone find compaction useful? I have always been a big believer of many small chats instead of large ones. There is certainly a time and a place where maybe I need to load in a lot to initial context BUT I often find that quality goes does as we near context limits and of course costs go massively up. I would rather repeat small notarized chunks than have a massive context window.

enforcing markdown files is much better. In a perfect world these docs are curated by humans, but if you can't be arsed then at least have some ground rules so these do not get flooded with context. Agents are pretty good at finding what they need for a task, if they're instructed correctly.

I stopped compacting and started making new chats every time and it has saved me an immense amount of context. I'm using deepseek and their cache hit is cheap, so combining this I don't spend more than 4 bucks a week.

I really don't mind keeping these docs up to date if it means I can leave my desk to go do something more important

Re: How Compaction Works in Pi

#90

Post by infecto »

ramon156 wrote: post #88
infecto wrote: post #86

Does anyone find compaction useful? I have always been a big believer of many small chats instead of large ones. There is certainly a time and a place where maybe I need to load in a lot to initial context BUT I often find that quality goes does as we near context limits and of course costs go massively up. I would rather repeat small notarized chunks than have a massive context window.

enforcing markdown files is much better. In a perfect world these docs are curated by humans, but if you can't be arsed then at least have some ground rules so these do not get flooded with context. Agents are pretty good at finding what they need for a task, if they're instructed correctly. I stopped compacting and started making new chats every time and it has saved me an immense amount of context. I'm using deepse…

Agree. I am a big believer in always starting new chats. The harness/model should be able to easily find anything we are working on.
Post reply on HN