Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built an OSS alternative to Azure OpenAI services

github.com

Post reply on HN
Print view View original

61–64 of 64 posts

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#61

Post by ij23 »

luyuanxin1995 wrote: post #41
aaronharnly wrote: post #39

Congratulations on shipping! We are currently evaluating replacing our homegrown version of an LLM proxy with this project: https://github.com/BerriAI/litellm Any comparison or contrast you would point out?

Litellm proxy is a pretty good project on its own. I am obviously biased because we are competitors. Here are my thoughts. * Litellm is declarative and it let you define everything in yaml * Bricks is not declarative and you control everything via API * Litellm does not have an UI * Bricks has a non open source UI * Litellm is written in python * Bricks is written in Golang * Litellm does not persist rate limits. The…

hi i'm the maintainer of litellm - we persist rate limits, they're written to a DB: https://docs.litellm.ai/docs/proxy/virtual_keys

- LiteLLM Proxy IS Exactly Compatible with the OpenAI SDK

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#62

Post by detente18 »

luyuanxin1995 wrote: post #41
aaronharnly wrote: post #39

Congratulations on shipping! We are currently evaluating replacing our homegrown version of an LLM proxy with this project: https://github.com/BerriAI/litellm Any comparison or contrast you would point out?

Litellm proxy is a pretty good project on its own. I am obviously biased because we are competitors. Here are my thoughts. * Litellm is declarative and it let you define everything in yaml * Bricks is not declarative and you control everything via API * Litellm does not have an UI * Bricks has a non open source UI * Litellm is written in python * Bricks is written in Golang * Litellm does not persist rate limits. The…

LiteLLM proxy (100+ LLMs in OpenAI format) is exactly compatible with the OpenAI endpoint. Here's how to call it with the openai sdk:

``` import openai

client = openai.OpenAI( api_key="anything", # proxy key - if set base_url="http://0.0.0.0:8000" # proxy url )

# request sent to model set on litellm proxy,

response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ { "role": "user", "content": "this is a test request, write a short poem" } ])

print(response)

``` Docs - https://docs.litellm.ai/docs/proxy/quick_start

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#63

Post by luyuanxin1995 »

luyuanxin1995 wrote: post #41
aaronharnly wrote: post #39

Congratulations on shipping! We are currently evaluating replacing our homegrown version of an LLM proxy with this project: https://github.com/BerriAI/litellm Any comparison or contrast you would point out?

Litellm proxy is a pretty good project on its own. I am obviously biased because we are competitors. Here are my thoughts. * Litellm is declarative and it let you define everything in yaml * Bricks is not declarative and you control everything via API * Litellm does not have an UI * Bricks has a non open source UI * Litellm is written in python * Bricks is written in Golang * Litellm does not persist rate limits. The…

Update: Litellm is compatible with OpenAI SDK
Post reply on HN