The CVE-2026-35029 privilege escalation in LiteLLM is a reminder that middleware is not something you can install once and forget.
What is your view on using self-hosted LLM gateway?
I wrote this post - https://leanroute.dev/blog/self-hosting-an-llm-gateway
Would like to hear your views.
Should I reconsider my decision of self-hosting LLM Gateway?
1–9 of 9 posts
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#2Isn't this a general class of problem that has a similar risk profile for any type of gateway that accepts inputs that are not very tightly bounded?
Not only LLM services.
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#3I'm running a gateway for my local LLMs, are you opening it to the internet? Who's going to craft a malicious payload?
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#4[dead]
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#5[flagged]
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#6[dead]
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#7[dead]
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#8Isn't this a general class of problem that has a similar risk profile for any type of gateway that accepts inputs that are not very tightly bounded? Not only LLM services.
yes, but LiteLLM was exposed to a CVE.
I switched to leanroute.dev
Re: Should I reconsider my decision of self-hosting LLM Gateway?
#9I'm running a gateway for my local LLMs, are you opening it to the internet? Who's going to craft a malicious payload?
was using LiteLLM for the company i work for. we were doing a poc.
But now we switched to leanroute.dev, we use our own provider keys. Gateway helps with the routing, caching, spend cap and monitoring usage.