11–20 of 39 posts
https://deeplearning4j.org https://github.com/deeplearning4j/ScalNet https://deeplearning4j.org/model-import-keras https://gitter.im/deeplearning4j/deeplearning4j
Isn't spark more versatile than tensorflow at this point? It does graph processing and deep learning. Plus it's built for distributed processing. Pyspark makes it easy to use.
Comparing Spark and TensorFlow is sort of like comparing Numpy and Pandas. There is some overlap, but they are pretty different things.
Spark is a big data manipulation tool, which comes with a somewhat-adequate machine learning library. TensorFlow is an optimised math library with machine learning operations built on it.
Spark doesn't support GPU operations (although as you note Databricks has proprietary extensions on their own cluster). DeepLearning4J and various other libraries do similar things.
However, if you are building your own Neural Network architectures then TF (which has highly optimised distributed training mode) is more useful.
Hmm, seems interesting but wondering how it compares to H2O's Sparkling Water. Been using that for clients and i love it.
Earlier quoted context omitted.
Spark doesn't do GPU acceleration, which is super important if you don't have a lot of spare CPU capacity. The one time I tried to train a DL model on CPU it was 48x slower, and with communication overhead it would have taken more than 48 cores to match the single GPH. And given you want to do hyperparameter search on top of that, those CPU cores start adding up.
that doesnt seem right. Databricks seems to have this in production. https://databricks.com/blog/2016/10/27/gpu-acceleration-in-d... https://databricks.com/blog/2016/12/21/deep-learning-on-data... I havent used this feature - but are you sure ?
Earlier quoted context omitted.
that doesnt seem right. Databricks seems to have this in production. https://databricks.com/blog/2016/10/27/gpu-acceleration-in-d... https://databricks.com/blog/2016/12/21/deep-learning-on-data... I havent used this feature - but are you sure ?
This seems exactly right. The Databricks GPU stuff isn't generally available and is built on TF anyway.
https://docs.databricks.com/user-guide/clusters/gpu.html
https://databricks.com/blog/2016/12/21/deep-learning-on-data...
In addition, IBM has multiple projects around GPU aware spark - https://github.com/IBMSparkGPU http://www.spark.tc/gpu-acceleration-on-apache-spark-2/
Deeplearning4j does this already. It has a huge community, a Scala API and does model import from Keras. It's important to note that Spark is not an efficient computation later -- it's best if used for fast ETL. If you get that wrong, your going to be training slow. https://deeplearning4j.org https://github.com/deeplearning4j/ScalNet https://deeplearning4j.org/model-import-keras https://gitter.im/deeplearning4j/deepl…
Earlier quoted context omitted.
Spark doesn't do GPU acceleration, which is super important if you don't have a lot of spare CPU capacity. The one time I tried to train a DL model on CPU it was 48x slower, and with communication overhead it would have taken more than 48 cores to match the single GPH. And given you want to do hyperparameter search on top of that, those CPU cores start adding up.
that doesnt seem right. Databricks seems to have this in production. https://databricks.com/blog/2016/10/27/gpu-acceleration-in-d... https://databricks.com/blog/2016/12/21/deep-learning-on-data... I havent used this feature - but are you sure ?
Earlier quoted context omitted.
This seems exactly right. The Databricks GPU stuff isn't generally available and is built on TF anyway.
hmm.. atleast databricks claims that the GPU clusters are beta, but generally available https://docs.databricks.com/user-guide/clusters/gpu.html https://databricks.com/blog/2016/12/21/deep-learning-on-data... In addition, IBM has multiple projects around GPU aware spark - https://github.com/IBMSparkGPU http://www.spark.tc/gpu-acceleration-on-apache-spark-2/
Yes, as I said elsewhere there are plenty of projects to enable GPU usage via Spark. Have you actually tried them though? I have (eg https://github.com/IBMSparkGPU/GPUEnabler/issues/25 ) and there are... issues.
Isn't spark more versatile than tensorflow at this point? It does graph processing and deep learning. Plus it's built for distributed processing. Pyspark makes it easy to use.
(Heavy Spark user here) Comparing Spark and TensorFlow is sort of like comparing Numpy and Pandas. There is some overlap, but they are pretty different things. Spark is a big data manipulation tool, which comes with a somewhat-adequate machine learning library. TensorFlow is an optimised math library with machine learning operations built on it. Spark doesn't support GPU operations (although as you note Databricks ha…
At google, their graph processing system (Expander) and deep learning framework (tensorflow) are separate systems. Spark looks to be built from the graph side (RDD) first and is now getting ML components.
how do you see spark evolving ?
Deeplearning4j does this already. It has a huge community, a Scala API and does model import from Keras. It's important to note that Spark is not an efficient computation later -- it's best if used for fast ETL. If you get that wrong, your going to be training slow. https://deeplearning4j.org https://github.com/deeplearning4j/ScalNet https://deeplearning4j.org/model-import-keras https://gitter.im/deeplearning4j/deepl…
I"ll do it for him.
I'd just like to say that as far as this niche is concerned. This is basically an attempt at "non gpus on spark".
We are heavily biased towards cuda and distributed gpu applications: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-...
I respect what intel is trying to do here, but it's going to take a lot more than "we built stuff" to get anyone to switch let alone build a community around.
To be fair to intel, I can't wait to see what they do with accelerators and phi, but I need to see more results first.
Competition in the space is definitely needed :D.
We have yet to see fpgas and nervana acquisition really play out as well.
It will take them a while to catch up either way.