Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An API for hosted deep learning models

blog.algorithmia.com

Post reply on HN
Print view View original

31–35 of 35 posts

Re: An API for hosted deep learning models

#31

Post by dharma1 »

flx42_ wrote: post #24
TaylorAlexander wrote: post #14

Earlier quoted context omitted.

Agreed. I brought up a system with nvidia-docker last week for a computer vision application and while it works, it seems fragile. There are more pieces than there should be and it seems easy to break. I also don't know if we can use multiple containers on one host, but it doesn't sound like it. It is not the final solution for containerized GPU applications.

Author of nvidia-docker here. You can definitely have multiple containers on each GPU if you want. If you find a bug or if you think the documentation was not great, please file a bug!

Thanks for your superb work. Is it possible to use nvidia-docker on several AWS instances, to use multiple GPUs? (To spread training on multiple GPUs for more speed and ram. Tensorflow and Caffe support distributed training but not sure if it's viable on dockerized envs on AWS?)

Re: An API for hosted deep learning models

#32

Post by flx42_ »

dharma1 wrote: post #31
flx42_ wrote: post #24

Earlier quoted context omitted.

Author of nvidia-docker here. You can definitely have multiple containers on each GPU if you want. If you find a bug or if you think the documentation was not great, please file a bug!

Thanks for your superb work. Is it possible to use nvidia-docker on several AWS instances, to use multiple GPUs? (To spread training on multiple GPUs for more speed and ram. Tensorflow and Caffe support distributed training but not sure if it's viable on dockerized envs on AWS?)

One container can use multiple GPUs on the same machine without problems.

For distributed training (which Caffe doesn't actually support, not the official version), you would have to run one container per instance, but this is more a configuration problem at the framework level, than a Docker or nvidia-docker problem.

Re: An API for hosted deep learning models

#33

Post by scottlocklin »

iraphael wrote: post #7
vonnik wrote: post #4

The history of machine learning startups is littered with companies that thought a hosted web service was a good idea. The problem with this model is that big data, by definition, is costly to move. So if a managed service is not generating and storing the data you need to process with machine learning or deep learning (as you might conceivably with AWS), then you probably don't want to move your data to those algori…

My understanding of the post is it would host learned models. You'd train them wherever, but host the learned model in algorithmia, which exposes it through an api, making it easy for others to use your model.

Because if I am capable of training a fancy pants deep learning model to do something helpgul, obviously I need a service to host my model so badly giving someone my model is a better idea than paying $10 a month for an EC2 instance.

Re: An API for hosted deep learning models

#34

Post by TaylorAlexander »

flx42_ wrote: post #24
TaylorAlexander wrote: post #14

Earlier quoted context omitted.

Agreed. I brought up a system with nvidia-docker last week for a computer vision application and while it works, it seems fragile. There are more pieces than there should be and it seems easy to break. I also don't know if we can use multiple containers on one host, but it doesn't sound like it. It is not the final solution for containerized GPU applications.

Author of nvidia-docker here. You can definitely have multiple containers on each GPU if you want. If you find a bug or if you think the documentation was not great, please file a bug!

Awesome. Thanks for the reply and I apologize for suggesting something incorrect.

It does strike me as tricky needing to match driver versions between the host and the container. Do you know if there is any effort to eliminate that requirement?

Also while we're chatting, is there any hope of NVIDIA open sourcing their linux drivers? How would such a move affect nvidia-docker?

Re: An API for hosted deep learning models

#35

Post by flx42_ »

TaylorAlexander wrote: post #34
flx42_ wrote: post #24

Earlier quoted context omitted.

Author of nvidia-docker here. You can definitely have multiple containers on each GPU if you want. If you find a bug or if you think the documentation was not great, please file a bug!

Awesome. Thanks for the reply and I apologize for suggesting something incorrect. It does strike me as tricky needing to match driver versions between the host and the container. Do you know if there is any effort to eliminate that requirement? Also while we're chatting, is there any hope of NVIDIA open sourcing their linux drivers? How would such a move affect nvidia-docker?

You don't need to match the driver version between the host and the container. Actually, you shouldn't include any driver file inside the container.

All the user-level driver-files required for execution are mounted when the container is started using a volume. This way you can deploy the same container on any machine with NVIDIA drivers installed.

We have more details on our wiki: https://github.com/NVIDIA/nvidia-docker/wiki/Internals

Concerning your last question: I don't have any information on this topic, but anyway it would not really impact nvidia-docker.

Post reply on HN