Apache Singa, a Distributed Deep Learning Platform
singa.incubator.apache.org
Apache Singa, a Distributed Deep Learning Platform
1–10 of 19 posts
Re: Apache Singa, a Distributed Deep Learning Platform
#2There are so many Apache projects these days (mostly seem to be in the incubator) - where do they all come from? I had once thought that Apache was a dumping ground for the technology of failed startups, but a lot of these projects seem to find new life in the incubator - so they must be doing something right.
Re: Apache Singa, a Distributed Deep Learning Platform
#3There are so many Apache projects these days (mostly seem to be in the incubator) - where do they all come from? I had once thought that Apache was a dumping ground for the technology of failed startups, but a lot of these projects seem to find new life in the incubator - so they must be doing something right.
The list of big name Apache projects is nuts, look at their project lists and you can easily pull out dozens of big projects and even incubator has a few popular projects.
Re: Apache Singa, a Distributed Deep Learning Platform
#4No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
Re: Apache Singa, a Distributed Deep Learning Platform
#5No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
if your data exceeds memory then a GPU is worthless. distributed makes it scalable
Re: Apache Singa, a Distributed Deep Learning Platform
#6No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
if your data exceeds memory then a GPU is worthless. distributed makes it scalable
It's not trivial to take any existing algorithm for training and scale it not only cross-GPU but also cross machine. Training tends to be very bandwith intensive. In fact that is the entire reason Nvidia is so heavily focused on this with their upcoming GPUs (Pascal). I am not saying it's not possible, just that it's a lot harder than "distributed makes it scalable".
Re: Apache Singa, a Distributed Deep Learning Platform
#7No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
if your data exceeds memory then a GPU is worthless. distributed makes it scalable
This is wrong. Training data can be streamed through GPU memory during training. It's your parameters that can't exceed GPU memory. You can get GPUs with 12 GB of memory, and they also support float16 so they can be twice as memory efficient as CPUs. If your model has more parameters than that, then you'll be waiting months or years for a single model to train using CPUs, even distributed.
Furthermore, almost any technique you use to distribute and scale training will work just as well regardless of whether the computations are happening on CPUs or GPUs.
Re: Apache Singa, a Distributed Deep Learning Platform
#8No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
It is in their schedule for december
http://singa.incubator.apache.org/develop/schedule.html
Re: Apache Singa, a Distributed Deep Learning Platform
#9And Cuban hackers everywhere chuckle at the unfortunate slang cuss word chosen as the name of this project... http://www.urbandictionary.com/define.php?term=Singa
Re: Apache Singa, a Distributed Deep Learning Platform
#10No GPU support => not interesting. GPUs are an order of magnitude faster. If you're doing deep learning on CPUs you're wasting your time and/or money.
It is in their schedule for december http://singa.incubator.apache.org/develop/schedule.html
It's only single node, multiple GPUs, though, which already exists in other frameworks.