Control LLM Spend and Access with any-LLM-gateway
blog.mozilla.ai
Control LLM Spend and Access with any-LLM-gateway
1–10 of 29 posts
Re: Control LLM Spend and Access with any-LLM-gateway
#2Thoughts on any-llm-gateway versus litellm-proxy?
litellm is a great library, but one team using litellm-proxy reported having many issues with it to me. I haven't tried it yet.
Re: Control LLM Spend and Access with any-LLM-gateway
#3There is also PydanticAI Gateway (https://ai.pydantic.dev/gateway/). I use it with the PydanticAI framework and it's quite nice.
Re: Control LLM Spend and Access with any-LLM-gateway
#4[flagged]
Re: Control LLM Spend and Access with any-LLM-gateway
#5[flagged]
Re: Control LLM Spend and Access with any-LLM-gateway
#6This service (llm proxy to all providers) are a dime-a-dozen
This one has very little on monitoring and no reference to OTEL in the docs
Re: Control LLM Spend and Access with any-LLM-gateway
#7Interested to see how this stacks up against Bifrost (fast but many features paywalled) and LiteLLM Proxy (featureful but garbage code quality). Especially if it gets a web admin / reporting frontend and high availability.
Re: Control LLM Spend and Access with any-LLM-gateway
#8I'm conflicted on what Mozilla is doing here. On the one hand, it is nice that they are getting involved but com'on, dont you all have Firefox to work on?
This is a classic case of an over enthusiastic engineer who says yes / raises hand to everything, but doesnt do any one thing properly. At some point, you have to sit down and tell them to focus on one thing and do it properly.
Re: Control LLM Spend and Access with any-LLM-gateway
#9Thoughts on any-llm-gateway versus litellm-proxy? litellm is a great library, but one team using litellm-proxy reported having many issues with it to me. I haven't tried it yet.
Yeah, I wonder what gaps in Litellm Proxy made Mozilla want to even do this.
Re: Control LLM Spend and Access with any-LLM-gateway
#10Interested to see how this stacks up against Bifrost (fast but many features paywalled) and LiteLLM Proxy (featureful but garbage code quality). Especially if it gets a web admin / reporting frontend and high availability.
We are just now looking into LLM Gateways and LiteLLM was one I was considering looking into. I’m curious to hear more about what makes the code quality garbage.