Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to use OpenRouter?

mmoustafa.com

Post reply on HN
Print view View original

91–100 of 208 posts

Re: So you want to use OpenRouter?

#91

Post by memoryleakgame »

I have been working on a product for months and for 1 of the specific models have become the number 3 user and very likely soon the number 2.

Its a google model. The edge cases are crazy to deal with and have taken a long time to find. I am also finding that since it has no fallbacks but no published rate limit I am single handedly taking the model down on what I thought were a reasonable amount of request. There is no other fallback that isn't google. I'm worried that with sustained usage from my users on launch in a few days. Clearly google can't be making that much money on it it im like 10% of the usage and the top 2-3 user of it spending several hundred a month in pre launch testing.

So why would they care to help a small time start up? I will have to jump to a cost effective different model and find the footguns all over again at somepoint.

Not sure where im going with this but just wanted to let others know

Re: So you want to use OpenRouter?

#93

Post by unscaled »

TZubiri wrote: post #27
bsaul wrote: post #5

That looks absolutely horrifying. What are the alternatives ??

Just use a single vendor. Literally nothing wrong with that, and you avoid the complexity of both n-1 of the vendors (leaving you with the highest quality vendor) as well as the issues with the aggregating layer. Not sure why people are drawn to this particular blunder. The promise of vendor neutrality maybe? I'll take working product over vendor-neutral slop anyways.

OP already answered that one:

They used a closed list of 3 vendors in prioritized order, and got 429ed out of two of them, while the third one stopped serving the mode.

This is less of a problem if you're running an agent locally and routing your problem to OpenRouter - you can pin to one or two models for consistency and just switch models when something goes bad. But the article is specifically about production traffic.

Re: So you want to use OpenRouter?

#94

Post by voakbasda »

memoryleakgame wrote: post #91

I have been working on a product for months and for 1 of the specific models have become the number 3 user and very likely soon the number 2. Its a google model. The edge cases are crazy to deal with and have taken a long time to find. I am also finding that since it has no fallbacks but no published rate limit I am single handedly taking the model down on what I thought were a reasonable amount of request. There is…

Building a product that depends on a Google product seems like folly at this point. They are notoriously unreliable about keeping products alive. I hope you don’t get burned.

Re: So you want to use OpenRouter?

#95

Post by memoryleakgame »

voakbasda wrote: post #94
memoryleakgame wrote: post #91

I have been working on a product for months and for 1 of the specific models have become the number 3 user and very likely soon the number 2. Its a google model. The edge cases are crazy to deal with and have taken a long time to find. I am also finding that since it has no fallbacks but no published rate limit I am single handedly taking the model down on what I thought were a reasonable amount of request. There is…

Building a product that depends on a Google product seems like folly at this point. They are notoriously unreliable about keeping products alive. I hope you don’t get burned.

Me too, im a bit nervous because I also have evals proving that flash models are actually getting worse at specific task on newer versions and one of my prime use cases peak around flash 3 and tanks in 3.8. So it would be very very horrible for me if they kill old flash models on open router.

Yay google, love living in fear of a vendor that they will rug pull me at any point because they can.

Re: So you want to use OpenRouter?

#97

Post by CubsFan1060 »

nhecker wrote: post #55
CubsFan1060 wrote: post #15

Another thing to be aware of -- apparently their billing limits don't work. Somehow someone was able to use my key from Singapore. It had a $10 daily limit and they were able to spend $100. Not only that, it got me blocked from all models for the frontier labs. On top of that, near as I can tell, there are no protections for your API key. No restrictions by country, IP, etc...

Just chiming in that I've never had this experience with them. I set, hit, and depend on those limits regularly. I would be wildly interested in reading what their support analysis had to say about this situation, if such a postmortem was done.

So far it was just "investigating". From the logs, my best guess is they sent all the requests at approximately the same time, and whatever limiting they do doesn't react fast enough.

I get it, the API key was my responsibility, but, setting the dollar limit is exactly the guard they suggest against that.

Re: So you want to use OpenRouter?

#98

Post by CubsFan1060 »

numlocked wrote: post #71
CubsFan1060 wrote: post #15

Another thing to be aware of -- apparently their billing limits don't work. Somehow someone was able to use my key from Singapore. It had a $10 daily limit and they were able to spend $100. Not only that, it got me blocked from all models for the frontier labs. On top of that, near as I can tell, there are no protections for your API key. No restrictions by country, IP, etc...

We do now indeed have IP restrictions for API keys

I think that's only on the enterprise plans? So no real protection for those of us using open router personally? Or did I miss other protections somewhere?

Re: So you want to use OpenRouter?

#99

Post by jwxz »

memoryleakgame wrote: post #91

I have been working on a product for months and for 1 of the specific models have become the number 3 user and very likely soon the number 2. Its a google model. The edge cases are crazy to deal with and have taken a long time to find. I am also finding that since it has no fallbacks but no published rate limit I am single handedly taking the model down on what I thought were a reasonable amount of request. There is…

For a Google model you might be better off directly using their API, which is unfortunately far more painful than using OpenRouter, but it might at least be more reliable for your product.

I think the rate limit issue happens because of OpenRouter sending so many requests to Google.

Re: So you want to use OpenRouter?

#100

Post by infecto »

embedding-shape wrote: post #51
bbor wrote: post #47

Earlier quoted context omitted.

If openrouter is only for people who "make their own benchmark" in a mission to roll something that's usually made by scientists with large budgets, it should say so and thus fade into deserved obscurity. I don't think anyone serious is just willy-nilly making individual requests against OpenRouter Despite your confidence, that is indeed the basis of this massive corporations entire business plan. If you just want be…

[flagged]

> Hmm, yeah, good and condense version of what my previous comment said. I'm much impressed by your reading ability.

10month old account with 20k karma. Low value rubbish postings as a professional user. Sad.

Post reply on HN