Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MCP server that reduces Claude Code context consumption by 98%

mksg.lu

Post reply on HN
Print view View original

71–80 of 119 posts

Re: MCP server that reduces Claude Code context consumption by 98%

#71

Post by blakec »

The FTS5 index approach here is right, but I'd push further: pure BM25 underperforms on tool outputs because they're a mix of structured data (JSON, tables, config) and natural language (comments, error messages, docstrings). Keyword matching falls apart on the structured half.

I built a hybrid retriever for a similar problem, compressing a 15,800-file Obsidian vault into a searchable index for Claude Code. Stack is Model2Vec (potion-base-8M, 256-dimensional embeddings) + sqlite-vec for vector search + FTS5 for BM25, combined via Reciprocal Rank Fusion. The database is 49,746 chunks in 83MB. RRF is the important piece: it merges ranked lists from both retrieval methods without needing score calibration, so you get BM25's exact-match precision on identifiers and function names plus vector search's semantic matching on descriptions and error context.

The incremental indexing matters too. If you're indexing tool outputs per-session, the corpus grows fast. My indexer has a --incremental flag that hashes content and only re-embeds changed chunks. Full reindex of 15,800 files takes ~4 minutes; incremental on a typical day's changes is under 10 seconds.

On the caching question raised upthread: this approach actually helps prompt caching because the compressed output is deterministic for the same query. The raw tool output would be different every time (timestamps, ordering), but the retrieved summary is stable if the underlying data hasn't changed.

One thing I'd add to Context Mode's architecture: the same retriever could run as a PostToolUse hook, compressing outputs before they enter the conversation. That way it's transparent to the agent, it never sees the raw dump, just the relevant subset.

Re: MCP server that reduces Claude Code context consumption by 98%

#73

Post by overfeed »

mksglu wrote: post #40
ZeroGravitas wrote: post #21

I've seen a few projects like this. Shouldn't they in theory make the llms "smarter" by not polluting the context? Have any benchmarks shown this effect?

That's the theory and it does hold up in practice. When context is 70% raw logs and snapshots, the model starts losing track of the actual task. We haven't run formal benchmarks on answer quality yet, mostly focused on measuring token savings. But anecdotally the biggest win is sessions lasting longer before compaction kicks in, which means the model keeps its full conversation history and makes fewer mistakes from l…

> When context is 70% raw logs and snapshots, the model starts losing track of the actual task

Which frontier model will (re)introduce the radical idea of separating data from executable instructions?

Re: MCP server that reduces Claude Code context consumption by 98%

#75

Post by jerf »

FuckButtons wrote: post #69
mksglu wrote: post #30

Earlier quoted context omitted.

That's pretty much the approach we took with context-mode. Tool outputs get processed in a sandbox, only a stub summary comes back into context, and the full details stay in a searchable FTS5 index the model can query on demand. Not trained into the model itself, but gets you most of the way there as a plugin today.

This is a partial realization of the idea, but, for a long running agent the proportion of noise increases linearly with the session length, unless you take an appropriately large machete to the problem you’re still going to wind up with sub optimal results.

Yeah, I'd definitely like to be able to edit my context a lot more. And once you consider that you start seeing things in your head like "select this big chunk of context and ask the model to simply that part", or do things like fix the model trying to ingest too many tokens because it dumped a whole file in that it didn't realize was going to be as large as it was. There's about a half-dozen things like that that are immediately obviously useful.

Re: MCP server that reduces Claude Code context consumption by 98%

#76

Post by wonnage »

FuckButtons wrote: post #64
esperent wrote: post #49

Earlier quoted context omitted.

Is it because of caching? If the context changes arbitrarily every turn then you would have to throw away the cache.

So use a block based cache and tune the block size to maximize the hit rate? This isn’t rocket science.

This seems misguided, you have to cache a prefix due to attention.

Re: MCP server that reduces Claude Code context consumption by 98%

#77

Post by wonnage »

esperent wrote: post #46
elephanlemon wrote: post #10

Earlier quoted context omitted.

Agree. I’d like more fine grained control of context and compaction. If you spend time debugging in the middle of a session, once you’ve fixed the bugs you ought to be able to remove everything related to fixing them out of context and continue as you had before you encountered them. (Right now depending on your IDE this can be quite annoying to do manually. And I’m not aware of any that allow you to snip it out if y…

> For example, if you’re working with a tool that dumps a lot of logged information into context I've set up a hook that blocks directly running certain common tools and instead tells Claude to pipe the output to a temporary file and search that for relevant info. There's still some noise where it tries to run the tool once, gets blocked, then runs it the right way. But it's better than before.

I think telling it to run those in a subagent should accomplish the same thing and ensure only the answer makes it to the main context. Otherwise you will still have some bloat from reading the exact output, although in some cases that could be good if you’re debugging or something

Re: MCP server that reduces Claude Code context consumption by 98%

#80

Post by lowbloodsugar »

nr378 wrote: post #17
elephanlemon wrote: post #10

Earlier quoted context omitted.

Agree. I’d like more fine grained control of context and compaction. If you spend time debugging in the middle of a session, once you’ve fixed the bugs you ought to be able to remove everything related to fixing them out of context and continue as you had before you encountered them. (Right now depending on your IDE this can be quite annoying to do manually. And I’m not aware of any that allow you to snip it out if y…

Oh that's quite a nice idea - agentic context management (riffing on agentic memory management). There's some challenges around the LLM having enough output tokens to easily specify what it wants its next input tokens to be, but "snips" should be able to be expressed concisely (i.e. the next input should include everything sent previously except the chunk that starts XXX and ends YYY). The upside is tighter context,…

So I built that in my chat harness. I just gave the agent a “prune” tool and it can remove shit it doesn’t need any more from its own context. But chat is last gen.
Post reply on HN