Viewing profile — chaoyu_

Check out BentoML https://github.com/bentoml

Most team don't really concern about the security side of things when using pickle for model deployment, more about performance and resource utilization. It works ok for very light…

Regardless of the architecture, you would still need the model server for online serving use cases. And you’re right that it’s best to decouple the validation/preprocessing logic f…

Exactly! A vertical solution focused on model serving and deployment would make more sense if it can easily integrate with the ML training platform for CICD

That’s is true, it would be more accurate to say FastAPI (or any ASGI) framework along is not enough for ML model serving, you need batching and runner for performance. And some ad…