Cog: Containers for Machine Learning
github.com
Cog: Containers for Machine Learning
1–10 of 15 posts
Re: Cog: Containers for Machine Learning
#2Woah, perfect timing! I was just about to start writing a dockerfile+fastapi wrapper for our newest ML project, will try this out instead!
Re: Cog: Containers for Machine Learning
#3This, here is going to save time in the order of (wo-)man lifetimes!
> No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.
I've personally spent more than a week of my life in sum sorting this out. It's really overdue someone stops the madness!
Re: Cog: Containers for Machine Learning
#4How does this compare to BentoML (https://www.bentoml.com/)?
Re: Cog: Containers for Machine Learning
#5Hello HN! One of the creators of Cog here.
We built this to deploy models to Replicate (https://replicate.com/), but it can also be used to deploy models to your own infra.
Andreas, my co-founder, used to work at Spotify. Spotify wanted to run models inside Docker containers, but Docker was too hard to use for most ML researchers. So, Andreas built a set of templates and scripts to help researchers deploy their own models.
This was mixed in with my experience working at Docker. I created Docker Compose, which makes Docker easier to use for dev environments. We were also joined by Zeke, who created Swagger (now OpenAPI), which is used to define a model’s inputs/outputs. Dominic and some other contributors have since joined! https://github.com/replicate/cog#contributors-
It’s still early days, so expect a few rough edges, but it’s ready to use for deploying models. We’d love to hear what you think.
Re: Cog: Containers for Machine Learning
#6How does this compare to BentoML ( https://www.bentoml.com/ )?
There is a fair bit of overlap.
Cog is optimized for getting a deep learning model inside a Docker image. We found that ML researchers struggled to use Docker, so we made that process easier. It generates a best practice Dockerfile with all your dependencies, and resolves the CUDA versions automatically. It also includes a queue worker, which we found was the optimal way of deploying long-running/batch models at Spotify and Replicate.
Bento is more flexible – the models can be used outside of Docker, and it has built-in support for deploying to lots of deployment environments, which Cog doesn't have yet.
Re: Cog: Containers for Machine Learning
#7Is there an easy way to create a similar dev environment for training on Linux that will take care of all the CUDA driver nonsense?
Re: Cog: Containers for Machine Learning
#8Woah, perfect timing! I was just about to start writing a dockerfile+fastapi wrapper for our newest ML project, will try this out instead!
Me too! About to start a new project that EXACTLY fits the bill for this tool... It sure sounds promising!
Re: Cog: Containers for Machine Learning
#9This is similar to build packs but maybe not as easy?
Re: Cog: Containers for Machine Learning
#10This, here is going to save time in the order of (wo-)man lifetimes! > No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you. I've personally spent more than a week of my life in sum sorting this out. It's really overdue someone stops the madness!
No cuda hell if you’re using pytorch. A single pip or conda command installs everything you need.