Cortex: Deploy machine learning models in production • Hacker News Board

Re: Cortex: Deploy machine learning models in production

#21

Post by sixhobbits » Fri, Oct 18, 2019, 9:29 AM UTC

isubasinghe wrote: post #20

This is basically my startup idea that I worked on for a while now ( https://aiscalr.isub.dev ) Looks like I am going to have to scrap that entire project now, seems pointless to keep working on it given how similar this is.

Similarity should be taken as validation, not a negative thing at all.

Re: Cortex: Deploy machine learning models in production

#22

Post by bermanoid » Fri, Oct 18, 2019, 10:22 AM UTC

kevinmershon wrote: post #2

Unfortunately, a somewhat popular Clojure library for machine learning on GitHub is also called Cortex, because this is going to make discussing machine learning APIs in the context of Clojure that much more confusing.

And I imagine many more machine learning tools will take the same name in the years to come, since it's about the most obvious one you could think of other than "brain".

Whatever is popular will survive...

Re: Cortex: Deploy machine learning models in production

#23

Post by vishalbollu » Fri, Oct 18, 2019, 12:38 PM UTC

solidasparagus wrote: post #19

ospillinger wrote: post #14

Earlier quoted context omitted.

Yes, Cortex uses ONNX Runtime ( https://github.com/microsoft/onnxruntime ) under the hood so any model that can be exported to ONNX can be deployed.

Is it only able to handle ONNX models? That's a pretty massive limitation compared to a hosted SageMaker endpoint.

Contributor here - Cortex supports Tensorflow saved models in addition to ONNX. PyTorch support is on the roadmap. Do you have specific frameworks in mind that you would like Cortex to support?

Re: Cortex: Deploy machine learning models in production

#24

Post by ovi256 » Fri, Oct 18, 2019, 2:09 PM UTC

isubasinghe wrote: post #20

This is basically my startup idea that I worked on for a while now ( https://aiscalr.isub.dev ) Looks like I am going to have to scrap that entire project now, seems pointless to keep working on it given how similar this is.

You should do customer development and find people willing to pay for your product.

If they're willing to pay, they'll even tell you why they can't use the open source tool.

Re: Cortex: Deploy machine learning models in production

#25

Post by TaupeRanger » Fri, Oct 18, 2019, 2:52 PM UTC

Stop naming things single word neuroscience terms. There are like 50 projects called "Cortex".

Re: Cortex: Deploy machine learning models in production

#26

Post by solidasparagus » Sat, Oct 19, 2019, 1:15 AM UTC

vishalbollu wrote: post #23

solidasparagus wrote: post #19

Earlier quoted context omitted.

Is it only able to handle ONNX models? That's a pretty massive limitation compared to a hosted SageMaker endpoint.

Contributor here - Cortex supports Tensorflow saved models in addition to ONNX. PyTorch support is on the roadmap. Do you have specific frameworks in mind that you would like Cortex to support?

Perfect. Nothing in particular other than TF.

Re: Cortex: Deploy machine learning models in production

#27

Post by ecnahc515 » Sat, Oct 19, 2019, 10:13 PM UTC

Not to be confused with weaveworks/CNCF Cortex project for high scale Prometheus monitoring https://github.com/cortexproject/cortex.

Re: Cortex: Deploy machine learning models in production

#28

Post by sails » Tue, Oct 22, 2019, 10:56 AM UTC

How does this compare to MLflow [0]?

Considering MLflow has a few components, I suppose you are building something closer to MLflow Models? How do they compare?

[0] https://mlflow.org/docs/latest/index.html

Re: Cortex: Deploy machine learning models in production

#29

Post by ospillinger » Wed, Oct 23, 2019, 11:07 PM UTC

sails wrote: post #28

How does this compare to MLflow [0]? Considering MLflow has a few components, I suppose you are building something closer to MLflow Models? How do they compare? [0] https://mlflow.org/docs/latest/index.html

From the MLflow Models docs: "An MLflow Model is a standard format for packaging machine learning models that can be used in a variety of downstream tools—for example, real-time serving through a REST API or batch inference on Apache Spark. The format defines a convention that lets you save a model in different “flavors” that can be understood by different downstream tools."

Cortex is what they are referring to as a downstream tool for real-time serving through a REST API. In other words, MLflow helps with model management and packaging, whereas Cortex is a platform for running real-time inference at scale. We are working on supporting more model packaging formats and I think it's a good idea to support the MLflow format as well.