221–228 of 228 posts
Co-founder and COO of OpenRouter here. Thanks everyone for the feedback here. Some of this we are aware of, some of it we aren't. Some we can fix, some of it is inherent to inference (and we in fact improve the situation dramatically). Philosophically, at OpenRouter we are trying to do two different things, that are sometimes at odds with one another: 1. Let you use a lot of capacity across a lot of providers, in a w…
Earlier quoted context omitted.
Thanks for the insight here. One thing to note about the first graph: nobody is doing as well as the first part on tool calling, and it's not close. This might be the fault of the other providers, but it's probably just something slightly different that the first party does with the model inference program than anybody else, and that's not sure to weights it's due to vLLM twiddling (or whatever) and probably becuase…
This matches what I found experimentally with gpt-oss-20b during OpenAI's red-teaming challenge. After moving from hosted inference to running the model myself on rented H100s via vast.ai, I saw the model refuse the same kinds of prompts at noticeably different rates depending on the inference stack — differences of roughly 5–10 percentage points with otherwise identical experimental parameters and seeds. So I very m…
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way. (And if this was a false positive, I apologize!)
I have been working on a product for months and for 1 of the specific models have become the number 3 user and very likely soon the number 2. Its a google model. The edge cases are crazy to deal with and have taken a long time to find. I am also finding that since it has no fallbacks but no published rate limit I am single handedly taking the model down on what I thought were a reasonable amount of request. There is…
You are hitting the google model through openrouter or directly?
That looks absolutely horrifying. What are the alternatives ??
I've been using Merge AI Gateway and it's been useful so far. They tend to add new models quickly, and support has been responsive. https://gateway.merge.dev/
Earlier quoted context omitted.
I've been using Merge AI Gateway and it's been useful so far. They tend to add new models quickly, and support has been responsive. https://gateway.merge.dev/
Glad you’ve been enjoying Gateway! We also have some huge discounts on open models like GLM 5.3 Flash.