Viewing profile — chaoyu

onnx is not a good option for LLM type of autoregressive generation

In my opinion, Automatic1111 is more of a development tool to experiment with different pipelines on your local GPU, or an internal tool serving a few users. OneDiffusion project a…

Check out BentoML, which is the underlying serving framework used by OpenLLM, and it supports other type of models and modality such as images and videos.

OpenLLM itself is under Apache 2 license, which does NOT restrict commercial use. However, OpenLLM as a framework can be extended to support other LLMs which may come with addition…

OpenLLM plan to provide an OpenAI-compatible API, which allows you to even use OpenAI's python client to talk to OpenLLM, user just need to change to Base URL to point to your Open…

Fine-tuning is coming up in the next release! You can actually try it out on the main branch :P

Looking forward to it! OpenLLM is adding a OpenAI-compatible API layer, which will make it even easier to migrate LLM apps built around OpenAI's API spec. Feel free to join our Dis…

Smaller models are likely more efficient to run inference and doesn't necessarily need the latest GPU. Larger language model trend to have better performance over more different ty…

The OpenLLM team is actively exploring those techniques for streamlining the fine-tuning process and making it accessible!

OpenLLM in comparison focuses more on building LLM apps for production. For example, the integration with LangChain + BentoML makes it easy to run multiple LLMs in parallel across …

BentoML.ai | ML Engineer, Backend Engineer | Full-time | Bay Area or Remote | Python, Kubernetes, MLOps platform, Data Infra, Tensorflow, PyTorch, etc BentoML is an open-source fra…

What does BentoML do? * Package models trained with any ML framework and reproduce them for model serving in production * Package once and deploy anywhere for real-time API serving…

BentoML.ai | Open Source Evangelist / Technical Writer | San Francisco or Remote | Full time or Contract | http://docs.bentoml.org/ BentoML is an open-source platform for high-perf…

I'm actually building a "modular open-source company/product" in the MLOps space: BentoML https://docs.bentoml.org/en/latest/

BentoML( https://github.com/bentoml/BentoML ) may help you with the process of building endpoints with both Deep learning models and logistic regression/tree models, and it automat…

hi Aaron, I'm one of the BentoML aurthors - great suggestion on pipreqs, will look into incorparating that into BentoML! It should be very straightforward adding support for saving…

Graphpipe solves a very unique problem when building ML model serving system, although BentoML is trying to solve a very different problem. We think it would be interesting to supp…

Our quick start guide notebook on Google Colab is also a great place to get started! https://colab.research.google.com/github/bentoml/BentoML/blo...

Thanks for sharing the project Kevlar1818. BentoML author here - we are building BentoML to empower Data Scientists to ship prediction services instead of delivering "models" to de…