Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to use OpenRouter?

mmoustafa.com

Post reply on HN
Print view View original

201–208 of 208 posts

Re: So you want to use OpenRouter?

#201

Post by ElectricalUnion »

numlocked wrote: post #169
jmward01 wrote: post #105

Yes to all this but more. The thing that made me leave and go to a single provider was token caching. I have to keep blocking providers that don't properly cache. I see performance tank and then I look in the logs and a new provider has been rotated in and every call to them is uncached because they are clearly broken. This has happened a few times now and essentially destroys cost savings (these providers also often…

We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see the "performance" area of our model pages. E.g. https://openrouter.ai/deepseek/deepseek-v4-flash-0731#perfor... Open to feedback on how to make this better.

Can I block providers (for model, not in general) that have sightly cheaper input/output tokens but have more that 10x the average cache cost?

Can I block providers (for a model, not in general) that set up cache write cost when the mode is free cache writes?

Re: So you want to use OpenRouter?

#202

Post by eli »

Spacemolte wrote: post #65
embedding-shape wrote: post #34

Earlier quoted context omitted.

> just incredibly unreliable and you are forced to pin providers and even then it can be a crapshoot as the author found. But that's the intention right? Even the name implies they just send stuff around for you, and if you want to control the routing, you'd lock down providers. I don't see how they could build what they wanted to build, and not have it end up unreliable if you freely round-robin between providers, i…

1. Yes, but if i can't rely on openrouter to route to providers to give me the best price and the best uptime across a number of providers, and it instead gives me inconsistent results, then I will not use them. 2. As long as the apis use the openai spec, it should be fine? And what makes you say the providers are using different weights? The blogpost says the exact opposite? 3. Great example, if amazon does not lead…

There’s a surprising amount of variety in between implementations of “OpenAI” endpoints.

Re: So you want to use OpenRouter?

#203

Post by drakmail »

artursapek wrote: post #88
joshstrange wrote: post #20

This squares with my, much much, smaller OpenRouter usage. It’s just incredibly unreliable and you are forced to pin providers and even then it can be a crapshoot as the author found. OpenRouter sells the idea of swapping being commodity providers but it couldn’t be further from the truth. Provider A is often not swappable for B or C (again, as this author found). It can be crazy-making as you sit there thinking “Ope…

I only use OpenRouter to benchmark models. For my production app, I have direct integrations with OpenAI, Anthropic, Google, xAI. Even just input caching is enough of a reason to do that, assuming you are trying to build the fastest and most cost-efficient product possible. I don't understand how people supposedly run production apps using OpenRouter unless those two variables somehow don't matter to them.

Not every usecase requires prompt caching, for example prompt enchantment jobs usually can't benefit from caching at all

Re: So you want to use OpenRouter?

#205

Post by owenshen24 »

MattyRad wrote: post #134

I agree that OpenRouter kinda sucks, but it took about 15 seconds to see that a few sentences got reverse-compressed into what is now this article, and it's slop.

> For an agent TAU is the score that matters and a 20 point swing is not noise.

+1, stuff like this is a big turnoff for reading

Re: So you want to use OpenRouter?

#206

Post by ozborn »

Former Openrouter user, unhappy to report that Openrouter will expire your tokens with very little warning with absolutely no business justification other than they can. If they were a bank and tokens money, their return is -100% interest after a year. Don't expect your tokens to be there for you after a year of no use....

Re: So you want to use OpenRouter?

#207

Post by dofm »

embedding-shape wrote: post #34
joshstrange wrote: post #20

This squares with my, much much, smaller OpenRouter usage. It’s just incredibly unreliable and you are forced to pin providers and even then it can be a crapshoot as the author found. OpenRouter sells the idea of swapping being commodity providers but it couldn’t be further from the truth. Provider A is often not swappable for B or C (again, as this author found). It can be crazy-making as you sit there thinking “Ope…

> just incredibly unreliable and you are forced to pin providers and even then it can be a crapshoot as the author found. But that's the intention right? Even the name implies they just send stuff around for you, and if you want to control the routing, you'd lock down providers. I don't see how they could build what they wanted to build, and not have it end up unreliable if you freely round-robin between providers, i…

> I guess OpenRouter is a bit like Amazon, in that they're just routing stuff around for you

This helps forecast one possible future for OpenRouter: they begin to offer in-house provision, and people begin to migrate their uses off the "marketplace" providers and onto the "fulfilled-by-us" provision.

Re: So you want to use OpenRouter?

#208

Post by randomblock1 »

numlocked wrote: post #169
jmward01 wrote: post #105

Yes to all this but more. The thing that made me leave and go to a single provider was token caching. I have to keep blocking providers that don't properly cache. I see performance tank and then I look in the logs and a new provider has been rotated in and every call to them is uncached because they are clearly broken. This has happened a few times now and essentially destroys cost savings (these providers also often…

We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see the "performance" area of our model pages. E.g. https://openrouter.ai/deepseek/deepseek-v4-flash-0731#perfor... Open to feedback on how to make this better.

You know how there's a router mode to use the cheapest provider? That only takes into account uncached rates, last I checked. Make another one that takes into account effective rates (the ones that include cache).
Post reply on HN