Viewing profile — rohaga

We have run it on DeepSWE, check out our blog post for details of how we leverage the intelligence of the smart models https://usetokenless.com/blog/building-tokenless/

Since we fan out in parallel, by definition the latency is only at worst the latency of the slowest model. The model in between is fairly light, so it seems to not materially affec…

Yeah this is a great observation--we have to collect some more data to understand new models as they come out. However, by having this composable architecture that separates "confi…

I agree that adjusting the reasoning effort to retain cache is a huge thing! But even doing that automatically is currently a challenge for people to figure out and do well, and co…

It automatically works with multi-turn routing. You can try it yourself in Claude Code or Codex

Thanks for the praise! We think the results are a work in progress. Routing is convincing for quality for the following reason. Take your favorite benchmark, and on each task, run …

Well, you can estimate the confidence BEFORE you start the task, too. That way you can restrict your trajectory to just a few models. We also think there are tons of people working…

Highly encourage you to read the blog post ( https://usetokenless.com/blog/building-tokenless ). Essentially, we estimate the confidence of a specific model failing or succeeding o…

Our research seems to indicate you can get a decent signal based on the first few reasoning trace summaries. We analyzed hundreds of billions of tokens to figure this out. Any open…

If we cut off an unconfident/expensive model early in thinking, then we don't have to pay for additional thinking cost at all. For many tasks, this is quite a large saving. That be…

> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the…

Hi HN, Rohit here from Tokenless ( https://usetokenless.com/ ), which I’m building alongside co-founders Andrew and Kev. We’re building an API gateway which routes agent traffic dy…