Confirmed YC orphan did not read Stripe news feed
Launch HN: Tokenless (YC S26) – Automatic model switching to save money
41–50 of 70 posts
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#42How is this different from OpenRouter?
I also developed something similar for my agent harness. It's based on model predictors and an exponential moving average:
https://github.com/rush86999/atom/blob/main/docs/architectur...
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#43Earlier quoted context omitted.
> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…
Seems like it might be more advantageous to just adjust reasoning effort to retain cache. Maybe in some exceptional cases where there will be a ton more inference to solve the problem, but going significantly dumber in that case seems counterintuitive. I can really only see the utility of things like spawning subagents to a lower tier model from another provider, and that's something harnesses can already handle (ie.…
Seems like it might be more advantageous to just adjust reasoning effort to retain cache.
Some agents when you alter the reasoning level, partially or complete wipe the cache. Never assume that changing reasoning is a no-impact change. I do not have the link but a French youtuber did a ton of research on this.
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#44So this only switches models if the cache is cold, because otherwise the economics of switching don't work. But most agentic work involves long strings of successive tool calls that benefit from a hot cache. Hot cache calls reduce input cost by 90%. This can basically only deliver cost savings in turns where the AI delivers a result to the user, the user waits at least 5 minutes (or the length of the cache), and then…
> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…
An approach that works really well for me in opencode is to use DeepSeek flash for the code exploration sub agent, and a smarter model for the overall task.
Code exploration tends to burn a ton of tokens and do not require frontier knowledge. It then returns a compact overview of the findings to the better model, that can therefore perform its task with less tokens.
Anyone else has a similar approach?
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#45[dead]
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#46Earlier quoted context omitted.
> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…
An approach that works really well for me in opencode is to use DeepSeek flash for the code exploration sub agent, and a smarter model for the overall task. Code exploration tends to burn a ton of tokens and do not require frontier knowledge. It then returns a compact overview of the findings to the better model, that can therefore perform its task with less tokens. Anyone else has a similar approach?
The solution that I am working on is designed to leverage cheap LLMs (think MiMo, GLM, etc.) to build cheap context to reduce tokens, but most importantly to keep the context clean.
https://github.com/gitsense/chat/tree/update-readme
The README and latest version is still being worked on but I am hoping to have the latest release in place soon.
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#47So this only switches models if the cache is cold, because otherwise the economics of switching don't work. But most agentic work involves long strings of successive tool calls that benefit from a hot cache. Hot cache calls reduce input cost by 90%. This can basically only deliver cost savings in turns where the AI delivers a result to the user, the user waits at least 5 minutes (or the length of the cache), and then…
> So this only switches models if the cache is cold Not exactly. It can also make sense if the cache is hot to switch models. For example, we might currently be on Deepseek and the next task is SO HARD that it doesn't make sense to not use frontier (1 turn to crack versus 100 turns for Deepseek to crack). Likewise, the router may judge that it's likely that the next few set of tool-calls will be ridiculously easy, an…
How would your classifier compare to the frontier model delegating to Deepseek via a MCP tool call itself? Doesn’t it have far more insight on the difficulty of the task then your router could? I’ve been doing this with Claude Code delegating to Codex and an Ollama Qwen model.
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#48So this only switches models if the cache is cold, because otherwise the economics of switching don't work. But most agentic work involves long strings of successive tool calls that benefit from a hot cache. Hot cache calls reduce input cost by 90%. This can basically only deliver cost savings in turns where the AI delivers a result to the user, the user waits at least 5 minutes (or the length of the cache), and then…
Pretty sure this is a start-up not an end-up. This is exactly what they will need to iterate on to drive costs down. And they’ll be in the perfect place to do it because they’ll have the telemetry. Exciting!
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#49Earlier quoted context omitted.
> the next task is SO HARD that it doesn't make sense to not use frontier How does tokenless quantify "so hard"? > 1 turn to crack versus 100 turns for Deepseek to crack Interesting definition for a "frontier". What is a "turn" here? Token count? Request count? Context-based? I've found that MiniMax M3 (a smaller model at 295b) will code up better when DeepSeek v4 Pro (1.6t) will not (and vice versa).
Highly encourage you to read the blog post ( https://usetokenless.com/blog/building-tokenless ). Essentially, we estimate the confidence of a specific model failing or succeeding on a specific task using our own foundation models. A turn here is a tool call/user input, anything that causes the model to get some new input. We're working on adding Minimax M3 and other models. We think that people have some intuitions a…
It will inspire more confidence if you say your classifier. Saying "your own foundation model" (charitably) suggests marketing hyperbole. If you're fine tuning a LLM to do this, at best it's going to be worse than a frontier model with suitably tuned skills (hence it should just be something in harness, not a service), and I wouldn't refer to that as "your own foundation model."
Re: Launch HN: Tokenless (YC S26) – Automatic model switching to save money
#50One big issue is prompt caching - how are you (or any other router) solving for it? The moment caching is gone real agentic workloads would spike like crazy in terms of cost.