Don't Put All Your Tokens in One Basket
theminimalistdeveloper.com
Don't Put All Your Tokens in One Basket
1–7 of 7 posts
Re: Don't Put All Your Tokens in One Basket
#2[deleted]
Re: Don't Put All Your Tokens in One Basket
#3How do you prepare the fallback if it already took tokens to query the simpler model?
Re: Don't Put All Your Tokens in One Basket
#4don't litellm and similar do this already?
Re: Don't Put All Your Tokens in One Basket
#5don't litellm and similar do this already?
Absolutely, a pretty good one.
Re: Don't Put All Your Tokens in One Basket
#6How do you prepare the fallback if it already took tokens to query the simpler model?
I don't know if I understood your question.
Re: Don't Put All Your Tokens in One Basket
#7don't litellm and similar do this already?
Actually, LiteLLM is the most popular, it was first product of this type, but there is plenty of more efficient and reliable alternate AI Gateways right now. I've written one - GoModel. https://gomodel.enterpilot.io/