Viewing profile — Aperswal

Currently building a LLM router and got a few of my startup friends to try it out. But, I feel like their advice is going to be slightly biased because they don't wanna hurt my fee…

Hey everyone, just added Bedrock and Vertex support to FlexInference. It now can do provider fallback on 429 and 5xx. Plus there is upstream retries you can setup via SDK. This way…

If you give an SLA of lets say 10 seconds to find cheaper inference. I look around different providers that offer flex/batch and try to fulfill in that time. If I can't then I esca…

The basic logic for default routing or flex routing?

This isn't open sourced, as of now. The routing is largely pass through or standard conversions between API surfaces. The SDK uses the OpenAPI specs for it's given API surface. The…

Hey everyone! Been working on a router that works with OpenAI, Anthropic, and Google (Gemini not Vertex), and just launched it for free. Let me know what you think.

Hey everyone! My name is Adi, I am the founder of FlexInference. I built this to reduce AI spend for all my side projects. I then turned it into a proxy for Anthropic, OpenAI, and …

Btw the link above is broken because of mismatched quotes But u can check out the repo at: https://github.com/TrySita/AutoDocs

Hey all, Adi and Sohan here. We're building Sita ( https://trysita.com/ ), an open-source documentation generator that builds a codebase-aware knowledge graph, writes documentation…