Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Smart model routing directly in Claude, Codex and Cursor

github.com

Post reply on HN
Print view View original

41–50 of 127 posts

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#41

Post by SoftTalker »

adchurch wrote: post #25
spqw wrote: post #10

This + making sure common requests are saved as reusable skills and scripts would probably save a large part of my token usage As prices increase we will see more of these tools to optimise and make the best use of token budget

100%, from what we've seen, for a lot of big companies that 1. don't have subsidized usage and 2. are pushing AI adoption hard, figuring out token costs is P0 or P1 for their eng leadership

So you're saying that since adopting AI/LLM tech many companies have their top engineering priority being optimizing the costs of that rather than ... addressing actual business needs?

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#42

Post by adchurch »

echelon wrote: post #40
alansaber wrote: post #19

Earlier quoted context omitted.

That is interesting, sounds like in practice you only end up routing between 2 models

Or not routing at all. In practice you just pick one and stick with it until the API stops or you hit performance issues.

The choice on the first turn is super important for this reason! But if a user prompt sends the convo in a very different direction then often it does make sense to reroute at that point.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#43

Post by bijowo1676 »

How come data privacy and confidentiality is not an issue with services like these?

Do people voluntarily use these proxies/routers, knowing their prompts, outputs and code will be seen by other people ?

I get it might be ok for personal projects, but for anything that makes money and is a part of business... this must be big no-no ?

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#44

Post by jakozaur »

It's rather hard to do at the proxy level with agentic coding, such as Claude Code or similar. These are long-chained sessions of tool use that heavily rely on prompt caching. Changing mid-flight is costly.

It looks like much more context is required to decide on the best model (e.g., summarizing logs might use a cheap model, whereas you likely want Opus/Mythos/GPT 5.6 to debug multithreading logic). In an agentic system, a decision about the model may be embedded in the decision to orchestrate the model.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#45

Post by adchurch »

SoftTalker wrote: post #41
adchurch wrote: post #25

Earlier quoted context omitted.

100%, from what we've seen, for a lot of big companies that 1. don't have subsidized usage and 2. are pushing AI adoption hard, figuring out token costs is P0 or P1 for their eng leadership

So you're saying that since adopting AI/LLM tech many companies have their top engineering priority being optimizing the costs of that rather than ... addressing actual business needs?

I guess delivering business value is always #1, I just meant it's the biggest problem they're trying to solve. Here's a recent example that was public: https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-c...

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#46

Post by victorbjorklund »

bijowo1676 wrote: post #43

How come data privacy and confidentiality is not an issue with services like these? Do people voluntarily use these proxies/routers, knowing their prompts, outputs and code will be seen by other people ? I get it might be ok for personal projects, but for anything that makes money and is a part of business... this must be big no-no ?

It is a router that runs locally.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#47

Post by mthoms »

adchurch wrote: post #7
stpedgwdgfhgdd wrote: post #2

The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial. How does this router translate to $$$ when developing?

You're right and that's why we built the router to be cache aware! Once it starts using one model, the threshold to switch to another model will be higher because the additional cost of the cache miss needs to be worth the cost savings or quality increase. This is the key thing that other routers we've seen miss: they're stateless so for a coding agent use case you end up spending more money due to all the cache miss…

This is a key point. I don't know if you can still edit your submission, but I think this would be helpful to mention up front. I'm looking forward to testing this.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#48

Post by adchurch »

jakozaur wrote: post #44

It's rather hard to do at the proxy level with agentic coding, such as Claude Code or similar. These are long-chained sessions of tool use that heavily rely on prompt caching. Changing mid-flight is costly. It looks like much more context is required to decide on the best model (e.g., summarizing logs might use a cheap model, whereas you likely want Opus/Mythos/GPT 5.6 to debug multithreading logic). In an agentic sy…

Yep cache awareness is super important, mentioned this in another thread here: (https://news.ycombinator.com/item?id=48689448)

But intuitively I think it makes sense that a model can learn what model to route things to if it has all the relevant info, and experimentally it works pretty well in our experience

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#50

Post by adchurch »

bijowo1676 wrote: post #43

How come data privacy and confidentiality is not an issue with services like these? Do people voluntarily use these proxies/routers, knowing their prompts, outputs and code will be seen by other people ? I get it might be ok for personal projects, but for anything that makes money and is a part of business... this must be big no-no ?

It's a real concern! We take this stuff super seriously (https://trust.mycroft.io/weave) and tbh most of our customers opt for the hosted version because it's much simpler on their end + they're already trusting us with a bunch of sensitive data.

But of course since the source is available you can also run it locally or self host

Post reply on HN