Show HN: Any-LLM – Lightweight router to access any LLM Provider • Hacker News Board

Show HN: Any-LLM – Lightweight router to access any LLM Provider

#1

Post by AMeckes » Tue, Jul 22, 2025, 5:40 PM UTC

We built any-llm because we needed a lightweight router for LLM providers with minimal overhead. Switching between models is just a string change : update "openai/gpt-4" to "anthropic/claude-3" and you're done.

It uses official provider SDKs when available, which helps since providers handle their own compatibility updates. No proxy or gateway service needed either, so getting started is pretty straightforward - just pip install and import.

Currently supports 20+ providers including OpenAI, Anthropic, Google, Mistral, and AWS Bedrock. Would love to hear what you think!

Show HN: Any-LLM – Lightweight router to access any LLM Provider
github.com

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#2

Post by sparacha » Tue, Jul 22, 2025, 6:21 PM UTC

There is liteLLM, OpenRouter, Arch (although that’s an edge/service proxy for agents) and now this. We all need a new problem to solve

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#3

Post by CuriouslyC » Tue, Jul 22, 2025, 6:26 PM UTC

sparacha wrote: post #2

There is liteLLM, OpenRouter, Arch (although that’s an edge/service proxy for agents) and now this. We all need a new problem to solve

LiteLLM is kind of a mess TBH, I guess it's ok if you just want a docker container to proxy to for personal projects, but actually using it in production isn't great.

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#4

Post by dlojudice » Tue, Jul 22, 2025, 6:32 PM UTC

I use Litellm Proxy, even in a dev environment via Docker, because the Usage and Logs feature greatly helps in providing visibility into LLM usage. The Caching functionality greatly helps in reducing costs for repetitive testing.

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#5

Post by dlojudice » Tue, Jul 22, 2025, 6:34 PM UTC

CuriouslyC wrote: post #3

sparacha wrote: post #2

There is liteLLM, OpenRouter, Arch (although that’s an edge/service proxy for agents) and now this. We all need a new problem to solve

LiteLLM is kind of a mess TBH, I guess it's ok if you just want a docker container to proxy to for personal projects, but actually using it in production isn't great.

> but actually using it in production isn't great.

I only use it in development. Could you elaborate on why you don't recommend using it in production?

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#6

Post by ieuanking » Tue, Jul 22, 2025, 6:37 PM UTC

sparacha wrote: post #2

There is liteLLM, OpenRouter, Arch (although that’s an edge/service proxy for agents) and now this. We all need a new problem to solve

we are trying to apply model-routing to academic work and pdf chat with ubik.studio -- def lmk what you think

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#7

Post by honorable_coder » Tue, Jul 22, 2025, 6:43 PM UTC

CuriouslyC wrote: post #3

sparacha wrote: post #2

There is liteLLM, OpenRouter, Arch (although that’s an edge/service proxy for agents) and now this. We all need a new problem to solve

LiteLLM is kind of a mess TBH, I guess it's ok if you just want a docker container to proxy to for personal projects, but actually using it in production isn't great.

the people behind envoy proxy built: https://github.com/katanemo/archgw - has the learnings of Envoy but natively designed to process/route prompts to agents and LLMs. Would be curious about your thoughts

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#8

Post by weinzierl » Tue, Jul 22, 2025, 6:47 PM UTC

Not to be confused with AnythingLLM.

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#9

Post by honorable_coder » Tue, Jul 22, 2025, 6:49 PM UTC

a proxy means you offload observability, filtering, caching rules, global rate limiters to a specialized piece of software - pushing this in application code means you _cannot_ do things centrally and it doesn't scale as more copies of your application code get deployed. You can bounce a single proxy server neatly vs. updating a fleet of your application server just to monkey patch some proxy functionality.

Re: Show HN: Any-LLM – Lightweight router to access any LLM Provider

#10

Post by RussianCow » Tue, Jul 22, 2025, 6:53 PM UTC

honorable_coder wrote: post #9

a proxy means you offload observability, filtering, caching rules, global rate limiters to a specialized piece of software - pushing this in application code means you _cannot_ do things centrally and it doesn't scale as more copies of your application code get deployed. You can bounce a single proxy server neatly vs. updating a fleet of your application server just to monkey patch some proxy functionality.

You can do all of that without a proxy. Just store the current state in your database or a Redis instance.