Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Smart model routing directly in Claude, Codex and Cursor

github.com

Post reply on HN
Print view View original

101–110 of 127 posts

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#101

Post by elgertam »

I ran into a problem at work recently: we are given access to a bunch of models up to a full Claude Opus 4.8, but a monthly budget of 100k tokens. We are also given access to Gemini 3.5 Flash & 3.1 Pro with a daily budget of 50M tokens, but no tool calling. I'd love to hook Claude Code (or Pi) into the Gemini model, but the lack of tool-calling makes it quite difficult. I've been planning out how an intelligent router might be able to use a token-efficient tool-calling model (including a small local open-weights model) to handle the basic tools like reading from the file system or interfacing with MCP servers such that context is gathered, but then send the built up context to the Gemini model where I have a nearly unlimited (for my use cases) token budget.

Could your router handle this?

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#102

Post by thombles »

elgertam wrote: post #101

I ran into a problem at work recently: we are given access to a bunch of models up to a full Claude Opus 4.8, but a monthly budget of 100k tokens. We are also given access to Gemini 3.5 Flash & 3.1 Pro with a daily budget of 50M tokens, but no tool calling. I'd love to hook Claude Code (or Pi) into the Gemini model, but the lack of tool-calling makes it quite difficult. I've been planning out how an intelligent route…

I’m curious how a workplace ends up with a model policy like this. It seems like you’d spend more time trying to work out how to use a tiny number of Opus tokens than doing it yourself.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#103

Post by Jaxkr »

elgertam wrote: post #101

I ran into a problem at work recently: we are given access to a bunch of models up to a full Claude Opus 4.8, but a monthly budget of 100k tokens. We are also given access to Gemini 3.5 Flash & 3.1 Pro with a daily budget of 50M tokens, but no tool calling. I'd love to hook Claude Code (or Pi) into the Gemini model, but the lack of tool-calling makes it quite difficult. I've been planning out how an intelligent route…

Monthly budget of 100k Opus tokens? So $2.50 worth?

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#105

Post by arendtio »

adchurch wrote: post #11
arendtio wrote: post #4

What is the difference from Cursors 'auto' mode?

Fun fact: Cursor's "auto" mode is just Composer (or at least it was last time I checked). So it's different in the sense that it actually does route to more than 1 model

How did you check? Like looking at the results or at the actual implementation?

I mean, I know that it mostly chooses Composer, but I wonder if it is hard-wired or if they have a logic that just selects Composer most of the time?

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#106

Post by forgeshiptoday »

Just curious how the router decides on which model to use. When I use Claude Code, I often ask Claude Code to decide itself if it should spawn a sub-agent to downgrade or upgrade the model. Claude Code is smart to know how much context and cache it has and will decide if it should use sub-agent with a lesser model (sometimes it costs more to re-fetch tokens with a Sonnet sub-agent if the parent agent already has the context).

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#108

Post by Lerc »

There are so many of these projects to wrangle AIs I think we might need an AI to go through, analysing each and amalgamating the good bits.

It makes me think of MakeFiles.

Make is sufficiently bad that everyone who has used it has considered writing a better way to do it. A good percentage of those people have done so.

On the other hand, make is also not so sufficiently bad that it cannot do its job. The choice becomes picking the thing that everyone has or one of the many many alternatives that proclaim their strengths and leave their weaknesses lurking to bite when they are least expected.

No single replacement to make dominates, and make lives on. I wonder if AI management is on a similar path.

Re: Show HN: Smart model routing directly in Claude, Codex and Cursor

#109

Post by port11 »

stpedgwdgfhgdd wrote: post #2

The thing I do not get with these routers is that you will have more cache misses (5min ttl). And if there is one thing i’ve learned; using the cache is crucial. How does this router translate to $$$ when developing?

Artefact-based workflows solve this problem, and I think it’s more effective to go in that direction.

I still have Claude Code because Opus makes good plans, but I hand the plan over to M3 on Pi with 99.9% cache hits on a long session. Lovely. Pi then makes a summary file that Opus can use to review the code/context.

But you do need them to write down their stuff, so that compaction and clear sessions can work off a nice, concise document.

And if you are simply using Claude Code, then /advisor is what you want: a sub-agent with a much cleaner context is spawned to handle something -> not cached per se, but much cheaper to run.

I’d stay away from workflows that automatically route between models unless you can afford the cache misses. That’s also why GLM 5.x is costing me much more, I don’t get good caching with it.

Post reply on HN