31–40 of 89 posts
I don't want some other tooling messing with my context. It's too important to leave to something that needs to optimize across many users, there by not being the best for my specifics. The framework I use (ADK) already handles this, very low hanging fruit that should be a part of any framework, not something external. In ADK, this is a boolean you can turn on per tool or subagent, you can even decide turn by turn or…
I guess I'm skeptical that this actually improves performance. I'm worried that the middle man, the tool outputs, can strip useful context that the agent actually needs to diagnose.
That's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon. Given our observations, the performance depends on the task and the model itself, most visible on long-running tasks
do you guys have any stats on how much faster this is than claude or codex's compression? claudes is super super slow, but codex feels like an acceptable amount of time? looks cool tho, ill have to try it out and see if it messes with outputs or not.
Earlier quoted context omitted.
That's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon. Given our observations, the performance depends on the task and the model itself, most visible on long-running tasks
How does the model know it needs more context?
We state this directly appended into the outputs so the model knows exactly where the lines were removed from.
I wonder what is the business model. It seems like the tool to solve the problem that won't last longer than couple of months and is something that e.g. claude code can and probably will tackle themselves soon.
Business model is: Get acquired
I wonder what is the business model. It seems like the tool to solve the problem that won't last longer than couple of months and is something that e.g. claude code can and probably will tackle themselves soon.
Business model is: Get acquired
The compression models are the product, not the proxy. The gateway is open-source because it's the distribution layer. Anthropic, Codex, and others are iterating on this too — but each only for their own agent. We're fully agent-agnostic and solely focused on compression quality, which is itself a hard problem that needs dedicated iteration.
Try it out and let us know how to make it better!
Is it similar to rtk? Where the output of tool calls is compressed? Or does it actively compress your history once in a while? If it's the latter, then users will pay for the entire history of tokens since the change uncached: https://platform.claude.com/docs/en/build-with-claude/prompt... How is this better?
This is a bit more akin to distill - https://github.com/samuelfaj/distill Advantage of SML in between some outputs cannot be compressed without losing context, so a small model does that job. It works but most of these solutions still have some tradeoff in real world applications.
Earlier quoted context omitted.
In addition to context rot, cost matters, I think lots of people use toke compression tools for that not because of context rot
From a determinism standpoint it might be better for the rot to occur at ingest rather than arbitrarily five questions later.
ok, its great
I can already prevent context pollution with subagents. How is this better?