Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Tokenless (YC S26) – Automatic model switching to save money

usetokenless.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#51

Post by ValentineC »

brandall10 wrote: post #27
rohaga wrote: post #3

Earlier quoted context omitted.

> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…

Seems like it might be more advantageous to just adjust reasoning effort to retain cache. Maybe in some exceptional cases where there will be a ton more inference to solve the problem, but going significantly dumber in that case seems counterintuitive. I can really only see the utility of things like spawning subagents to a lower tier model from another provider, and that's something harnesses can already handle (ie.…

Claude's cache breaks with a reasoning effort adjustment:

https://platform.claude.com/docs/en/build-with-claude/effort...

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#52

Post by judge2020 »

rohaga wrote: post #3
mediaman wrote: post #2

So this only switches models if the cache is cold, because otherwise the economics of switching don't work. But most agentic work involves long strings of successive tool calls that benefit from a hot cache. Hot cache calls reduce input cost by 90%. This can basically only deliver cost savings in turns where the AI delivers a result to the user, the user waits at least 5 minutes (or the length of the cache), and then…

> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…

Would it not also be more efficient to promot to delegate more tasks where possible to just less capable agents? This is basically what Cursor does (although most certainly in an effort to get more grok/composer usage)

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#53

Post by yiyingzhang »

This approach only works for small context requests. For large context and relatively smaller output (say understanding a huge code base), the cost will mainly be on prefill, and sending the large context to multiple models will only increase the cost, possibly by some factor.

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#55

Post by renezander030 »

The failure mode I would want addressed before putting this in front of an agent fleet is silent quality regression. When a turn gets routed to a cheaper model and the agent still completes the task, the trace reads as a success, and nobody attributes the worse output to the routing decision until it has compounded downstream. In the enterprise pipelines I build the spend win rarely came from per-turn model choice anyway, it came from cutting how many turns hit an LLM at all, with models pinned per workflow step so a regression was attributable to one change. That pinning is what made the system auditable, which is what a buyer's platform team actually signs off on. Do you run a shadow sample where routed turns are also executed on the frontier model, so a customer can see the measured quality delta instead of trusting the router's own confidence estimate?

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#58

Post by myshapeprotocol »

Optimizing resource allocation and cost-efficiency through dynamic model switching is a smart response to scaling AI workloads. When building data-intensive applications and managing automated pipelines, keeping operational overhead lean and modular is key to sustainable engineering

Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money

#59

Post by rvz »

rush86999 wrote: post #42

How is this different from OpenRouter? I also developed something similar for my agent harness. It's based on model predictors and an exponential moving average: https://github.com/rush86999/atom/blob/main/docs/architectur...

There is no difference.

It is just another metoo copy of OpenRouter, but "Backed by Y Combinator™"

Post reply on HN