61–70 of 73 posts
I strongly recommend the authors rewrite the readme by hand. It’s kind of a snif test for how much care someone put into this project.
I have the same complaint about oh-my-pi's readme. The tone is obnoxious. It's somehow jaded and matter-of-fact at the same time. Like it was written by that one guy at work who never misses a chance to brag about how clever he is.
"Without the Bloat"
vs.
"litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC"
Seriously anybody considering 100k+ LOC not a bloat? You made my day!
Let's just say the author's and my definition of bloat is not the same. Full disclosure, I'm the guy who reimplemented etcher (over 400Mb) in a mere 300Kb, Capstone (over 1Mb) in only 66Kb and who compressed LPC charactersheets (over 700Mb) into 4Mb. That's my interpretation of "non-bloated".
LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs.
Btw, you should update httpx to httpx2, and I think it's not much effort to remove openai's SDK compatibility.
I'd love to have a provider-agnostic LLM router (almost) dependency free (aside from httpx2).
This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept. The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.
THIS! And it's way cheaper than others like Kong =)
ROTFL, I had to laugh so hard! "Without the Bloat" vs. "litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC" Seriously anybody considering 100k+ LOC not a bloat? You made my day! Let's just say the author's and my definition of bloat is not the same. Full disclosure, I'm the guy who reimplemented etcher (over 400Mb) in a mere 300Kb, Capstone (over 1Mb)…
This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept. The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.
Earlier quoted context omitted.
We run it at my org and it's never been a noticeable resource hog. It's actually the best performer between it, our AI observability stack and the front end.
It may not be a huge resource hog, but it adds a ton of latency. https://www.getmaxim.ai/bifrost/resources/benchmarks Having ran both LiteLLM and Bifrost for months, I can largely confirm the numbers from those benchmarks for myself.
That's not to say Bifrost wouldn't have been better, but the choice to use LiteLLM was arrived at after a fair bit of internal discussion (most of which predated my addition to the team), and so far we've seen nothing from LiteLLM that has been contradictory to the pros/cons they thought would be the case when LiteLLM was adopted.
Or in other words, the org will be happy indeed when they have solved so many of the rest of the problems we've had in AI uptake that the difference in latency between one AI gateway or the other becomes a problem to be solved.
ROTFL, I had to laugh so hard! "Without the Bloat" vs. "litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC" Seriously anybody considering 100k+ LOC not a bloat? You made my day! Let's just say the author's and my definition of bloat is not the same. Full disclosure, I'm the guy who reimplemented etcher (over 400Mb) in a mere 300Kb, Capstone (over 1Mb)…
You misunderstood. This new project has 2,900 LOC. Maybe the spelling change is too subtle.
I wrote an entire https client from ground up in 117 LOC (and in a low level language, not Python): https://gitlab.com/bztsrc/skrellm/-/blob/main/src/https.c
Again, this guy and me disagree on what "not a bloat" means.
Earlier quoted context omitted.
You misunderstood. This new project has 2,900 LOC. Maybe the spelling change is too subtle.
No, you forgot about the dependencies (others said it to be 700Mb). And 2,900 LOC is still way too much for handling a remote API call, especially in a high level language such as Python where the biggest part of the task is delegated to separate modules. I wrote an entire https client from ground up in 117 LOC (and in a low level language, not Python): https://gitlab.com/bztsrc/skrellm/-/blob/main/src/https.c Again,…
On the contrary, the readme for this project, `litelm`, is contrasting with the bloat project, `litellm`. This project subty deboats an L.
The bloat (or featuritis) project has the 700mb and 100k+ LOC, this de-bloat has 2k LOC and specifies its only two deps ones most people want as deps (although httpx should be swapped to its successor).
So this response is off base, the 700Mb isn't forgotten, it's been dropped.