Earlier quoted context omitted.
The trick is to rarely switch, or switch at task boundaries. Often the conclusion of routing is actually "this one model is actually at the pareto front for this task, just use it always".
But then it's better to just not have a gateway switch models at all. Just have the harness able to choose which model its sub-agents use, then tell it how to split up tasks and which models to use when doing so.
Show HN: We built open OpenRouter that turns usage into a better model
11–20 of 50 posts
Re: Show HN: We built open OpenRouter that turns usage into a better model
#12Re: Show HN: We built open OpenRouter that turns usage into a better model
#13Amazing! Really brilliant idea, thank you for sharing this project. There is so much ground to cover in the LLM gateway / routing / reporting world, and this is a great start. The Tinker implementation is my favorite part, fine tuning is much better than a sea of context files.
Re: Show HN: We built open OpenRouter that turns usage into a better model
#14Re: Show HN: We built open OpenRouter that turns usage into a better model
#15>The gateway adds under 1 ms for BYOK requests Amazing! Really brilliant idea, thank you for sharing this project. There is so much ground to cover in the LLM gateway / routing / reporting world, and this is a great start. The Tinker implementation is my favorite part, fine tuning is much better than a sea of context files.
Re: Show HN: We built open OpenRouter that turns usage into a better model
#16Open source and no markup is the right default for a gateway. The caching question above is the one I would want answered before swapping models though.
Re: Show HN: We built open OpenRouter that turns usage into a better model
#17Re: Show HN: We built open OpenRouter that turns usage into a better model
#18Re: Show HN: We built open OpenRouter that turns usage into a better model
#19You started it a week ago? I look forward to checking back in 3 weeks when you've exited for $1B