Viewing profile — santiago-pl

I came up with this idea of artificially slowing down expensive models within a company to incentivize people to use cheaper ones. I'm curious - has anyone tried this at scale?

GoModel author here. I prepared self-reproducible benchmarks and published them on my blog. Might be helpful. I don't think latency is the best metric for comparing AI gateways, as…

If you experience any issues with LiteLLM, you may try GoModel - the AI Gateway I'm working on. It consumes ~60x less resources and is more reliable :)

Actually, LiteLLM is the most popular, it was first product of this type, but there is plenty of more efficient and reliable alternate AI Gateways right now. I've written one - GoM…

I'm working on the last AI gateway you'll ever try - GoModel :) I recently created a benchmark and it looks like GoModel is the fastest and most lightweight open-source (self-hoste…

I’ve created an honest, reproducible benchmark of self-hosted AI gateways literally two days ago. I posted it on HN here: https://news.ycombinator.com/item?id=48688213 To be totall…

I mentally treat it as part of the documentation right now.

Currently, I want to keep them close to the documentation, where I link to them directly. The fewer repositories I have, the easier it is to maintain them.

Thank you! Feel free to reach out to me over Discord in the case of any feedback or feature request.

GoModel author is here. This is my attempt to honestly compare GoModel vs LiteLLM vs Portkey vs Bifrost AI gateways.

Giorgi is the semantic caching master at GoModel right now. Let me ping him, and he'll get back to you here.

I'll definitely take a look at GAI myself! I like this beaver(?) at README.

[dead]

I'm wroking on it full-time right now. It might be challenging, especially when it comes to interactions with video, audio, and image models. I'm just trying to stay on top of what…

My thoughts about this: Benchmarking AI gateways properly is harder than it looks. Feature sets differ meaningfully - exact vs semantic caching, cluster mode, guardrails, audit log…

I've released a new version of GoModel (0.1.20) with explicit support for vllm. You can now use it even with a few vLLM instances. Like this: docker run --rm -p 8080:8080 \ -e VLLM…

That's great news! The AI model ecosystem is changing so fast.

TBH I decided to write GoModel because I needed something like this for my startup, enterpilot, and LiteLLM didn’t meet my needs.

I'll consider it for sure.

It's like fuel costs in a supply chain. When you buy apples at the store, you don't think about oil prices. But if trucks ran on something cheaper, more efficient, or less taxed, t…

"... and I don't see if I would be able to track usage from individual end-users through a header". Currently we have a unified concept of User-Paths. Once you add a specific heade…

The LiteLLM SDK is intentionally on the website. You can "talk" to GoModel with it because both projects use an OpenAI-compatible API under the hood. You can use it like this: from…

Yes, the number of meaningful providers might be around 20-30.