Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hello, Tensorflow

oreilly.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Hello, Tensorflow

#31

Post by tromobne8vb »

netinstructions wrote: post #23
tromobne8vb wrote: post #13

As I've been reading about tensorflow lately I feel like I'm missing something regarding distributed processing. How can Tensorflow 'scale up' easily if you are outside of Google? We have big datasets that I want to run learning on but it seems awkward to do with tensorflow. We're big enough that the team managing our cluster is separate than development and it is a huge pain if we need them to go install tools on ea…

Have you read through https://www.tensorflow.org/versions/master/how_tos/distribut... ? Since version 0.8 of TensorFlow they've had a way to do distributed processing. Their blog post in April mentioned it - https://research.googleblog.com/2016/04/announcing-tensorflo... That said, I haven't actually attempted any distributed processing, but it looks possible. If anyone has actually tried it and can speak to it I wou…

I have read that. I even re-read it before making my post.

That implementation requires starting individual tasks on each node in your cluster.

>To create a cluster, you start one TensorFlow server per task in the cluster. Each task typically runs on a different machine, but you can run multiple tasks on the same machine (e.g. to control different GPU devices).

I'm used to using tools that can roll out to a cluster with more finesse than that. The Spark wrapper seems to provide some capability to do this automatically, but even the Spark wrapper requires installing python libraries on each node.

Re: Hello, Tensorflow

#32

Post by gradys »

anilshanbhag wrote: post #24

I like the clarity of thought and structure of the article. I have used Tensorflow and had to explain it to a friend. So many times, I end up assuming things which are obvious to me but not to someone getting started. As said in the article, Tensorflow stands out for the ease of use and is to the best of my knowledge first distributed learning framework. Theano, Torch et al are faster but do not come with goodies lik…

There are different notions of "fast" as well. Theano might be faster on a single GPU (not actually sure if this is still true), but TensorFlow can be distributed to multiple GPUs in one machine or even across several machines. Several groups have reported speedups that are near-linear in the number of GPUs you distribute across. This can make TensorFlow orders of magnitude faster than Theano.

Re: Hello, Tensorflow

#33

Post by tanlermin »

tromobne8vb wrote: post #31
netinstructions wrote: post #23

Earlier quoted context omitted.

Have you read through https://www.tensorflow.org/versions/master/how_tos/distribut... ? Since version 0.8 of TensorFlow they've had a way to do distributed processing. Their blog post in April mentioned it - https://research.googleblog.com/2016/04/announcing-tensorflo... That said, I haven't actually attempted any distributed processing, but it looks possible. If anyone has actually tried it and can speak to it I wou…

I have read that. I even re-read it before making my post. That implementation requires starting individual tasks on each node in your cluster. >To create a cluster, you start one TensorFlow server per task in the cluster. Each task typically runs on a different machine, but you can run multiple tasks on the same machine (e.g. to control different GPU devices). I'm used to using tools that can roll out to a cluster w…

would something like conda or anaconda help?

https://docs.continuum.io/anaconda-cluster/index https://www.continuum.io/blog/developer-blog/getting-most-ou...

Re: Hello, Tensorflow

#34

Post by RockyMcNuts »

50CNT wrote: post #14

TensorFlow is admirably easier to install than some other frameworks I thought most frameworks are fairly easy to install in python, usually with a single call to pip. NLTK takes one "pip install nltk" and then "python", "import nltk", "nltk.download()" to download all the corpuses and miscellaneous data. Installing tensorflow seems complicated compared to that. # Ubuntu/Linux 64-bit, CPU only: $ sudo pip install --u…

once you have to install a GPU device driver, CUDA, tensorflow itself, related python packages like numpy, and you have choices of native installer, Linux packages, pip, conda, there's potential to find yourself in dependency hell, all of those layers are evolving rapidly.

I found these helpful (on AWS)

- http://ramhiser.com/2016/01/05/installing-tensorflow-on-an-a...

- http://tleyden.github.io/blog/2015/11/22/cuda-7-dot-5-on-aws...

of course you don't need to install CUDA just to learn, can run tensorflow on CPU only, but part of the point of the graph paradigm is to design a computation and offload it to GPU.

Re: Hello, Tensorflow

#36

Post by ajschumacher »

bduerst wrote: post #35

This is fantastic - thank you for doing this. When paired with the browser tool it makes a lot more sense: http://playground.tensorflow.org/ Is this planned to be released as an intro in a book about tensorflow?

Thanks! There aren't any firm plans, but we are thinking about what might make sense.

Re: Hello, Tensorflow

#37

Post by chronic80 »

gradys wrote: post #32
anilshanbhag wrote: post #24

I like the clarity of thought and structure of the article. I have used Tensorflow and had to explain it to a friend. So many times, I end up assuming things which are obvious to me but not to someone getting started. As said in the article, Tensorflow stands out for the ease of use and is to the best of my knowledge first distributed learning framework. Theano, Torch et al are faster but do not come with goodies lik…

There are different notions of "fast" as well. Theano might be faster on a single GPU (not actually sure if this is still true), but TensorFlow can be distributed to multiple GPUs in one machine or even across several machines. Several groups have reported speedups that are near-linear in the number of GPUs you distribute across. This can make TensorFlow orders of magnitude faster than Theano.

TensorFlow is second in speed, right behind torch (fp16). Theano is quite far behind at this point.

Re: Hello, Tensorflow

#38

Post by bhntr3 »

tromobne8vb wrote: post #31
netinstructions wrote: post #23

Earlier quoted context omitted.

Have you read through https://www.tensorflow.org/versions/master/how_tos/distribut... ? Since version 0.8 of TensorFlow they've had a way to do distributed processing. Their blog post in April mentioned it - https://research.googleblog.com/2016/04/announcing-tensorflo... That said, I haven't actually attempted any distributed processing, but it looks possible. If anyone has actually tried it and can speak to it I wou…

I have read that. I even re-read it before making my post. That implementation requires starting individual tasks on each node in your cluster. >To create a cluster, you start one TensorFlow server per task in the cluster. Each task typically runs on a different machine, but you can run multiple tasks on the same machine (e.g. to control different GPU devices). I'm used to using tools that can roll out to a cluster w…

Yeah, I'm trying to figure this out too. TensorFlow needs Yarn support. Ideally, Yarn would allocate resources and inform the processes of the various workers in the graph, etc. etc. I see that as the harder part. If you use mesos, then there is some preliminary support for that. https://github.com/tensorflow/tensorflow/issues/1996

Since TensorFlow has native dependencies on CUDA stuff for GPU support, I don't think there's much of a way to get around installing things on every machine. You might be able to package a python env without CUDA for spark to run using conda. Here's an interesting blog post about that: https://www.continuum.io/blog/developer-blog/conda-spark

But I'm not sure I see the point in running TensorFlow without GPU support. And if you're hoping to run GPU machines on an existing spark cluster and intelligently allocate the GPU stuff to the right machine. . . that's gonna be tough. Here's an interesting talk on that from the last spark summit: https://www.youtube.com/watch?v=k6IOWblLQK8&feature=youtu.be

Ultimately, you're probably better off just running your own gpu cluster strictly for your TensorFlow model on ephemeral AWS spot instances.

Or just use Google Cloud Machine Learning. That's what Google wants and expects you to do anyway. Borg is the Borg. You will be assimilated.

Re: Hello, Tensorflow

#39

Post by nl »

tromobne8vb wrote: post #31
netinstructions wrote: post #23

Earlier quoted context omitted.

Have you read through https://www.tensorflow.org/versions/master/how_tos/distribut... ? Since version 0.8 of TensorFlow they've had a way to do distributed processing. Their blog post in April mentioned it - https://research.googleblog.com/2016/04/announcing-tensorflo... That said, I haven't actually attempted any distributed processing, but it looks possible. If anyone has actually tried it and can speak to it I wou…

I have read that. I even re-read it before making my post. That implementation requires starting individual tasks on each node in your cluster. >To create a cluster, you start one TensorFlow server per task in the cluster. Each task typically runs on a different machine, but you can run multiple tasks on the same machine (e.g. to control different GPU devices). I'm used to using tools that can roll out to a cluster w…

I think what is going on here is that what we see as complications are actually features, but that doesn't become clear until you are operating with your NNs in production, at scale.

What you want to be able to do is control which devices (CPUs, GPUs or co-processors[1]) execute which part of your model (eg, GPU for training, co-processors for inference, who knows what else).

Yahoo released some code to deal with similar issues, but with Caffe on YARN[2].

[1] https://cloudplatform.googleblog.com/2016/05/Google-supercha...

[2] http://yahoohadoop.tumblr.com/post/129872361846/large-scale-...

Re: Hello, Tensorflow

#40

Post by nl »

bhntr3 wrote: post #38
tromobne8vb wrote: post #31

Earlier quoted context omitted.

I have read that. I even re-read it before making my post. That implementation requires starting individual tasks on each node in your cluster. >To create a cluster, you start one TensorFlow server per task in the cluster. Each task typically runs on a different machine, but you can run multiple tasks on the same machine (e.g. to control different GPU devices). I'm used to using tools that can roll out to a cluster w…

Yeah, I'm trying to figure this out too. TensorFlow needs Yarn support. Ideally, Yarn would allocate resources and inform the processes of the various workers in the graph, etc. etc. I see that as the harder part. If you use mesos, then there is some preliminary support for that. https://github.com/tensorflow/tensorflow/issues/1996 Since TensorFlow has native dependencies on CUDA stuff for GPU support, I don't think…

TensorFlow without GPU support is very useful for inference.
Post reply on HN