What is the difference from Cursors 'auto' mode?
Fun fact: Cursor's "auto" mode is just Composer (or at least it was last time I checked). So it's different in the sense that it actually does route to more than 1 model
Show HN: Smart model routing directly in Claude, Codex and Cursor
11–20 of 127 posts
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#12Isn't this more expensive than always using the same model, since, as I understand, by routing to different models you give up on cache?
If you statelessly route each new request: yes it does end up being more expensive!
So our routing is cache-aware. It will have a much higher threshold to switch from one model to another if there's already some cache for the first model. Experimentally this solves the problem (like I said we've saved 40% ourselves vs. what we would have otherwise paid).
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#13Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using. I'm not convinced it would route to the right model based on my diction or whatever.
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#14but this means you work with API pricing rather than subscription pricing. Isn’t it better to use claude or codex CLI etc directly in terms of cost?
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#15It is funny. We are building something similar.
Oh cool, feel free to reach out to me at andrew@workweave.ai if you ever want to share notes! We've learned a lot in the process of building this so far :)
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#16The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial. How does this router translate to $$$ when developing?
[deleted]
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#17but this means you work with API pricing rather than subscription pricing. Isn’t it better to use claude or codex CLI etc directly in terms of cost?
If you have a Claude/Codex subscription then we use that (and account for the subsidized price accordingly when making routing decisions) instead of API billing. So you get the best of both worlds: subsidized usage for frontier models + save by using open/smaller models when it's genuinely better.
In practice, lots of ppl are using this to make their Claude sub limits go further!
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#18"We reward the routing model when it selects an LLM that achieves the task successfully" sounds pretty oversimplified
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#19The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial. How does this router translate to $$$ when developing?
You're right and that's why we built the router to be cache aware! Once it starts using one model, the threshold to switch to another model will be higher because the additional cost of the cache miss needs to be worth the cost savings or quality increase. This is the key thing that other routers we've seen miss: they're stateless so for a coding agent use case you end up spending more money due to all the cache miss…
That is interesting, sounds like in practice you only end up routing between 2 models
Re: Show HN: Smart model routing directly in Claude, Codex and Cursor
#20Man, I'm not so sure if I'd use something like this because the way I prompt already changes based upon what model I am using. I'm not convinced it would route to the right model based on my diction or whatever.
Yep this was always the reason to avoid "auto" mode in cursor.