Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Litelm: LiteLLM Without the Bloat

github.com

Post reply on HN
Print view View original

1–10 of 73 posts

Re: Litelm: LiteLLM Without the Bloat

#4

Post by Centigonal »

This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept.

The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.

Re: Litelm: LiteLLM Without the Bloat

#6

Post by OutOfHere »

Centigonal wrote: post #4

This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept. The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.

LiteLLM doesn't quite live up to its name. With all those features, there is nothing "lite" about it. It is essential for a project to live up to its name.

Imagine Sqlite adding heavy features from Postgresql, e.g. row-level security.

Re: Litelm: LiteLLM Without the Bloat

#7

Post by freshtake »

First off, cool project! It's always great to see derivatives that question the efficiency of the established product.

I think the main thing the readme is missing is the core benefits. Reducing LOC and dependencies is cool, but it would be great to understand if this provides some additional benefits like lower latency or memory requirements.

Re: Litelm: LiteLLM Without the Bloat

#9

Post by josephh »

LeBit wrote: post #8

How does it compare to Bifrost?

I'm always confused by LLM proxies that claim to support tool calling. Even for Bifrost that claims to be doing it, at least when I was checking it out, I found out that while it injects the list of MCP tools that's available on the proxy-side, it doesn't actually make the call on client's behalf, and clients get confused by it (response returns MCP call request whose tool doesn't exist on the client-side).
Post reply on HN