Cortex – An ML model deployment platform that runs in your AWS account
1–10 of 25 posts
Re: Cortex – An ML model deployment platform that runs in your AWS account
#2A comparison with AWS's own offering (SageMaker Model Serving) would be helpful.
Re: Cortex – An ML model deployment platform that runs in your AWS account
#3Has anyone used this? It looks useful!
Re: Cortex – An ML model deployment platform that runs in your AWS account
#4A comparison with AWS's own offering (SageMaker Model Serving) would be helpful.
We solve a similar problem to SageMaker but we are focused on developer experience and flexibility.
- Deployments are defined with declarative configuration and no custom Docker images are required (although you can use your own if you want)
- You have full access to the instances, autoscaling groups, security groups, etc
- Less tied to AWS (GCP support is in the works)
- We are working on higher level features like prediction monitoring, alerting, and model retraining
- It's open source and free vs SageMaker's ~40% markup
Re: Cortex – An ML model deployment platform that runs in your AWS account
#5You should be using Terraform as your configuration management language. Please do not invent your own. Pretty much your yaml is a pseudo-configuration management language.
That will become one of the biggest blockers because of the amount of automation that already exists.
By leveraging Terraform, you will also have the added benefit of getting all the other pieces of AWS/GCP/azure components for free - and is rock stable and production tested.
Re: Cortex – An ML model deployment platform that runs in your AWS account
#6You should be using Terraform as your configuration management language. Please do not invent your own. Pretty much your yaml is a pseudo-configuration management language. That will become one of the biggest blockers because of the amount of automation that already exists. By leveraging Terraform, you will also have the added benefit of getting all the other pieces of AWS/GCP/azure components for free - and is rock…
Thanks for the feedback! We aren't trying to invent another infrastructure provisioning language, and I agree that Terraform would be the right choice if that was the case. Our YAML is more similar to the configuration of deployment tools like Netlify or CircleCI. We use CloudFormation and Kubernetes under the hood but our goal is to provide a much higher abstraction for data scientists / ML engineers.
Re: Cortex – An ML model deployment platform that runs in your AWS account
#7You should be using Terraform as your configuration management language. Please do not invent your own. Pretty much your yaml is a pseudo-configuration management language. That will become one of the biggest blockers because of the amount of automation that already exists. By leveraging Terraform, you will also have the added benefit of getting all the other pieces of AWS/GCP/azure components for free - and is rock…
Thanks for the feedback! We aren't trying to invent another infrastructure provisioning language, and I agree that Terraform would be the right choice if that was the case. Our YAML is more similar to the configuration of deployment tools like Netlify or CircleCI. We use CloudFormation and Kubernetes under the hood but our goal is to provide a much higher abstraction for data scientists / ML engineers.
Not entirely. The abstractions are different between infrastructure deployment.... versus configuration yml of circleci.
The declaration of deployment state is a very BIG and hard problem that has had millions of collective man hours spent over decades. I urge you not to think of it as a simple configuration.
In fact it is so hard that AWS has to build a new language on top of typescript ..versus cloudformation templates that it already had.
https://docs.aws.amazon.com/cdk/latest/guide/home.html
What you are building makes sense - I would drop cloudformation and surface Terraform right till the top.
So the way to use your tool is to install and use a new Terraform "provider".
Re: Cortex – An ML model deployment platform that runs in your AWS account
#8Compare and contrast with Seldon on Kubeflow?
Re: Cortex – An ML model deployment platform that runs in your AWS account
#9Compare and contrast with Seldon on Kubeflow?
My understanding is that Seldon and Kubeflow are more geared towards infrastructure engineers. Our goal is to hide the infrastructure tooling so that Kuberentes, Docker, or AWS expertise isn’t required. Cortex installs with one command, models are deployed with minimal declarative configuration, autoscaling works by default, and you don’t need to build Docker images / manage a registry.
Re: Cortex – An ML model deployment platform that runs in your AWS account
#10Earlier quoted context omitted.
Thanks for the feedback! We aren't trying to invent another infrastructure provisioning language, and I agree that Terraform would be the right choice if that was the case. Our YAML is more similar to the configuration of deployment tools like Netlify or CircleCI. We use CloudFormation and Kubernetes under the hood but our goal is to provide a much higher abstraction for data scientists / ML engineers.
Not entirely. The abstractions are different between infrastructure deployment.... versus configuration yml of circleci. The declaration of deployment state is a very BIG and hard problem that has had millions of collective man hours spent over decades. I urge you not to think of it as a simple configuration. In fact it is so hard that AWS has to build a new language on top of typescript ..versus cloudformation templ…
The Terraform provider idea is interesting, I'll think about it more carefully. Almost all of our deployment configuration under the hood is done with Kubernetes (which is focused on the declaration of deployment state). We modeled our configuration after Kubernetes for that reason, and we want to go beyond low-level infrastructure configuration by allowing users to configure prediction tracking, model retraining thresholds, and other more ML specific features using the same declarative paradigm and in the same configuration files.