Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to use OpenRouter?

mmoustafa.com

Post reply on HN
Print view View original

161–170 of 208 posts

Re: So you want to use OpenRouter?

#161

Post by jwxz »

neilmovva wrote: post #140
jwxz wrote: post #103

I also believe some providers fallback to another model entirely. I was recently using Kimi K3 and saw that some requests had no reasoning trace whatsoever. Unsurprisingly, those requests were routed to the less reputable providers (Sail Research).

hi, I'm one of the founders of Sail. I'm very sorry that you had a bad experience with us! We are serious about serving models correctly, and always publish a link to the exact HF checkpoint we're using for each model in our docs. If you ever have an issue like this again, please send a note to support@sailresearch.com and we'll make it right with a detailed postmortem.

Hey, thanks for reaching out. Just wanted to share my experience. I really thought there was an issue in your API.

I had quite a few Kimi K3 requests served by Sail. Most of these were fine and had the reasoning traces (I like reading them), however I noticed that some requests were being generated unusually quick and had no reasoning trace, which led me to believe another fallback model was being used, since Kimi K3 always emits reasoning tokens.

Re: So you want to use OpenRouter?

#162

Post by roger_maddux_iv »

numlocked wrote: post #79

Co-founder and COO of OpenRouter here. Thanks everyone for the feedback here. Some of this we are aware of, some of it we aren't. Some we can fix, some of it is inherent to inference (and we in fact improve the situation dramatically). Philosophically, at OpenRouter we are trying to do two different things, that are sometimes at odds with one another: 1. Let you use a lot of capacity across a lot of providers, in a w…

[dead]

Re: So you want to use OpenRouter?

#163

Post by msp26 »

fzysingularity wrote: post #111

The author’s comments on vision providers is especially interesting. We saw that most providers don’t provide native video url support, have high-variability in vision performance (likely due to the fact that they’re serving different quantization levels behind the same model id). If you’re building vision-native apps, there are so many footguns in vLLM/SGLang serving configurations, let alone the routing/orchestrati…

Do you have any advice on this front? I use vllm for a project but only for text tasks at the moment.

Re: So you want to use OpenRouter?

#165

Post by numlocked »

frenchtoast8 wrote: post #120
numlocked wrote: post #113

Earlier quoted context omitted.

Hmm, let me check. That certainly seems wrong. Can you send me an email w/ your email so I can look into it? im cc at openrouter.ai. Or DM me on X? x.com/cclark

My ticket number is #107156 (opened 9 days ago). The point is that the Discord is full of people not getting responses, and I'm not even the one waiting the longest. The only help they get is from the AI assistant in the server saying don't bump the ticket or you'll be punished by getting moved to the back of the queue. So there's nothing anybody can do but wait who knows how long.

Yeah we are very under-staffed. We are hiring aggressively. I readily admit we have inadequate support staffing right now, but we are going as fast as we can to ramp up.

Re: So you want to use OpenRouter?

#167

Post by numlocked »

noahbp wrote: post #150
numlocked wrote: post #79

Co-founder and COO of OpenRouter here. Thanks everyone for the feedback here. Some of this we are aware of, some of it we aren't. Some we can fix, some of it is inherent to inference (and we in fact improve the situation dramatically). Philosophically, at OpenRouter we are trying to do two different things, that are sometimes at odds with one another: 1. Let you use a lot of capacity across a lot of providers, in a w…

I have attempted to report numerous errors in your Chat UI, but you seem to have decided that customer input is undesired, as you have an AI support system that ignores my problem and refuses to escalate to anyone or create a ticket. To this day, your in-chat “Report an Issue” button still does not work consistently, and I am still billed for empty responses from many image providers.

Will look into this.

Re: So you want to use OpenRouter?

#168

Post by mmoustafa »

johnsmith1840 wrote: post #159

Same experience. I migrated over and am happy but that intro was brutal. My basics are I have a test suite that: 1. Finds newest models of my versions 2. Inferences every single model and a few providers for each with a short problem 3. Analyze latency and if a model failed the stupid simple questions drop it and the provider 4. Run larger context haystack kinds of problems. It's cheap and fast less than 5$ so I can…

Yes, you hit it on the head. It's worth it, but you have to do your own evaluation and promotion.

Example I forgot to mention: `:nitro` ranking is not the fastest, I do a round robin sampling with representative payloads to find out the fastest providers and reorder my list on the fly.

Re: So you want to use OpenRouter?

#169

Post by numlocked »

jmward01 wrote: post #105

Yes to all this but more. The thing that made me leave and go to a single provider was token caching. I have to keep blocking providers that don't properly cache. I see performance tank and then I look in the logs and a new provider has been rotated in and every call to them is uncached because they are clearly broken. This has happened a few times now and essentially destroys cost savings (these providers also often…

We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see the "performance" area of our model pages. E.g. https://openrouter.ai/deepseek/deepseek-v4-flash-0731#perfor...

Open to feedback on how to make this better.

Re: So you want to use OpenRouter?

#170

Post by numlocked »

radicality wrote: post #124
SXX wrote: post #121

Open router have one more annoying disadvantage: credits you buy expire in around 1 year +/- ~30 days. They do sent warning email 30 days in advance, but then just make them disappear. If you for some reason had unusee balance and forget about it; its just gone.

Oh, interesting, I feel like a year so ago I read something posted from openrouter team that they write that credits expire, but that they in actuality don’t expire them. But maybe I’m misremembering, or perhaps that’s changed for the worse in last few months :S

We started actually expiring the credits a ~month ago. If you make any kind of API request, it resets the clock. We try to make it a very generous policy, but we can't keep a monotonically increasing liability on the books. We end up owing (a lot) of taxes on it, but can't actually recognize revenue. We would much rather you spend the credits! Hence the reminder emails, and generous "clock reset" policy.
Post reply on HN