[flagged]
Show HN: Context Gateway – Compress agent context before it hits the LLM
21–30 of 89 posts
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#22Is it similar to rtk? Where the output of tool calls is compressed? Or does it actively compress your history once in a while? If it's the latter, then users will pay for the entire history of tokens since the change uncached: https://platform.claude.com/docs/en/build-with-claude/prompt... How is this better?
This is a bit more akin to distill - https://github.com/samuelfaj/distill
Advantage of SML in between some outputs cannot be compressed without losing context, so a small model does that job. It works but most of these solutions still have some tradeoff in real world applications.
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#23Funny enough, Anthropic just went GA with 1m context claude that has supposedly solved the lost-in-the-middle problem.
In addition to context rot, cost matters, I think lots of people use toke compression tools for that not because of context rot
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#24Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#25I guess I'm skeptical that this actually improves performance. I'm worried that the middle man, the tool outputs, can strip useful context that the agent actually needs to diagnose.
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#26I guess I'm skeptical that this actually improves performance. I'm worried that the middle man, the tool outputs, can strip useful context that the agent actually needs to diagnose.
That's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon.
Given our observations, the performance depends on the task and the model itself, most visible on long-running tasks
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#27Is it similar to rtk? Where the output of tool calls is compressed? Or does it actively compress your history once in a while? If it's the latter, then users will pay for the entire history of tokens since the change uncached: https://platform.claude.com/docs/en/build-with-claude/prompt... How is this better?
We do both:
We compress tool outputs at each step, so the cache isn't broken during the run. Once we hit the 85% context-window limit, we preemptively trigger a summarization step and load that when the context-window fills up.
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#28Funny enough, Anthropic just went GA with 1m context claude that has supposedly solved the lost-in-the-middle problem.
Just for anyone else who hadn't seen the announcement yet, this Anthropic 1M context is now the same price as the previous 256K context - not the beta where Anthropic charged extra for the 1M window: https://x.com/claudeai/status/2032509548297343196 As for retrieval, the post shows Opus 4.6 at 78.3% needle retrieval success in 1M window (compared with 91.9% in 256K), and Sonnet 4.6 at 65.1% needle retrieval in 1M (co…
[deleted]
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#29I wonder what is the business model. It seems like the tool to solve the problem that won't last longer than couple of months and is something that e.g. claude code can and probably will tackle themselves soon.
Business model is: Get acquired
Re: Show HN: Context Gateway – Compress agent context before it hits the LLM
#30Funny enough, Anthropic just went GA with 1m context claude that has supposedly solved the lost-in-the-middle problem.
In addition to context rot, cost matters, I think lots of people use toke compression tools for that not because of context rot
From a determinism standpoint it might be better for the rot to occur at ingest rather than arbitrarily five questions later.