Learning to Learn in TensorFlow
github.com
Learning to Learn in TensorFlow
1–10 of 22 posts
Re: Learning to Learn in TensorFlow
#2Can someone explain to me the benefits of using TensorFlow over Theano?
Re: Learning to Learn in TensorFlow
#3Can someone explain to me the benefits of using TensorFlow over Theano?
[deleted]
Re: Learning to Learn in TensorFlow
#4"Learning to learn by gradient descent by gradient descent"
Re: Learning to Learn in TensorFlow
#5Can someone explain to me the benefits of using TensorFlow over Theano?
I've heard that Tensorflow is built to take advantage of multiple GPUs automatically, whereas Theano (by default at least) can only make use of a single GPU.
Re: Learning to Learn in TensorFlow
#6"Learning to learn by gradient descent by gradient descent" https://arxiv.org/abs/1606.04474
Other equally exciting papers that relates to learning to learn in DL.
"Neural Architecture Search with Reinforcement Learning"
https://arxiv.org/abs/1611.01578
"RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning"
https://arxiv.org/abs/1611.02779
"Designing Neural Network Architectures using Reinforcement Learning"
Re: Learning to Learn in TensorFlow
#7Can someone explain to me the benefits of using TensorFlow over Theano?
I've heard that Tensorflow is built to take advantage of multiple GPUs automatically, whereas Theano (by default at least) can only make use of a single GPU.
TF is still single-thread, massive problem for real concurrency.
What TF can do is take one graph and distribute the work among many GPU's or CPU's.
Massive Scaling problems, I think over 5 Cpu's TF begins to degrade
https://arimo.com/machine-learning/deep-learning/2016/arimo-...
*
Problem is TF was not written for distributed systems, it would need to be rewritten from scratch. One rumor is google made TF public, because its obsolete, in house they have rewritten a new product, but their not going to share it.
*
It's google and people like google, TF is a wrapper, there is even a wrapper for the wrapper called "TFLearn" that is much easier to use than TF.
What is best? Depends upon what you like Lua, python, C++, amd or nvidia, intel or amd, once you tie yourself to language and hardware you only have a few choices.
Not much of anything is new we weren't doing in Mathematica 8 years ago. Then it was free to download via piratebay, and worked just fine with nvidia hw, now its 2017 and you can get it for free from google, and it supports the same nvidia hw.
*
Learn them all, they all have pluses & minuses, sadly I would say 90% of the code examples on github for ML are broken, or don't work, thus you need to spend months learning which works. Another good reason for 'tflearn' it all works, and the stuff from Montreal works.
Re: Learning to Learn in TensorFlow
#8Video related to library...
Nando de Freitas - Learning to Learn, to Program, to Explore and to Seek Knowledge (NIPS 2016)
Re: Learning to Learn in TensorFlow
#9Video related to library... Nando de Freitas - Learning to Learn, to Program, to Explore and to Seek Knowledge (NIPS 2016) https://www.youtube.com/watch?v=tPWGGwmgwG0
Same topic presented at KDD, with timestamp that cuts to the results.
Re: Learning to Learn in TensorFlow
#10Can someone explain to me the benefits of using TensorFlow over Theano?
I've heard that Tensorflow is built to take advantage of multiple GPUs automatically, whereas Theano (by default at least) can only make use of a single GPU.
"Automatically" isn't the best word, as TensorFlow won't make use of multiple GPUs unless you explicitly tell it to (at this time). That said, there are a number of benefits to using TensorFlow (including the ability to use multiple GPUs, if not automatically :) )
- Several common gradient optimization algorithms (Momentum, AdaGrad, AdaDelta, Adam, etc) are implemented already, which makes it a bit faster to get your training logic in place
- Going along with the above, there is more in the TensorFlow API focused specifically on training models, as opposed to being purely a math engine. Some might consider the extra funtionality "bloat", but I think it serves a good purpose
- The afforementioned multi-GPU functionality is nice, once you get used to it. It's good for either training multiple versions of a model in parallel or doing data parallel updates of parameters
- There are tools for compiling your trained models as static C++ binaries on mobile devices
- The TensorFlow ecosystem is quite nice: TensorBoard for visualizing training, the topology of your model, and various statistics (most recently visualizing projections of embeddings). TensorFlow Serving for deploying trained models. TF Slim for a more Keras-like layer by layer approach to model building. Several pre-trained models to jump start your own work.
- No compile times. There is a "no optimizations" option in Theano to remove the compilation, but many people's experience with Theano is having to wait to iterate on their code.
- I think the community is pretty swell too :) The Google team does a good job of responding to and working with folks who open issues or PRs
Generally, I'd say TensorFlow is really good when you want to minimize the amount of time between researching, training, and deploying your model.
Edit: line formatting