Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why TensorFlow instead of Theano for deep learning?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#31

Post by dimatura »

DLEnthusiast wrote: post #7
visarga wrote: post #4

TensorFlow is better for deployment. Pytorch is better for research. Theano/Keras is simpler to use and a little faster than TensorFlow

"PyTorch is better for research" is a weird, unsubstantiated statement. The fact is that few serious researchers use PyTorch (and even those complain about it). It's mostly grad students in a handful of labs. The only researchers I know who use PyTorch have been from FaceBook, and that's because they were implicitly forced to use it (PyTorch is developed by FaceBook). According to https://medium.com/@karpathy/icml-ac…

As long as you're citing @karpathy, "I've been using PyTorch a few months now and I've never felt better. I have more energy. My skin is clearer. My eye sight has improved." (https://twitter.com/karpathy/status/868178954032513024).

My two cents as a researcher who has used theano, caffe, pytorch and TF: they all have their pros and cons. After starting out with theano, I really appreciate the dynamic nature of pytorch: makes debugging and exploration easier compared to the static frameworks. Researchers tend to value these features over deployability, scalability and raw speed (though pytorch is no slouch). So I fully expect pytorch to get a lot of momentum in the near future.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#33

Post by shoo »

dimatura wrote: post #31
DLEnthusiast wrote: post #7

Earlier quoted context omitted.

"PyTorch is better for research" is a weird, unsubstantiated statement. The fact is that few serious researchers use PyTorch (and even those complain about it). It's mostly grad students in a handful of labs. The only researchers I know who use PyTorch have been from FaceBook, and that's because they were implicitly forced to use it (PyTorch is developed by FaceBook). According to https://medium.com/@karpathy/icml-ac…

As long as you're citing @karpathy, "I've been using PyTorch a few months now and I've never felt better. I have more energy. My skin is clearer. My eye sight has improved." ( https://twitter.com/karpathy/status/868178954032513024 ). My two cents as a researcher who has used theano, caffe, pytorch and TF: they all have their pros and cons. After starting out with theano, I really appreciate the dynamic nature of pyto…

> @karpathy, "I've been using PyTorch a few months now and I've never felt better. I have more energy. My skin is clearer. My eye sight has improved."

http://www.oneweirdkerneltrick.com/

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#34

Post by rryan »

Anyone who loves the Lisp concept of "code is data" will love TensorFlow.

Instead of coding imperatively, you write code to build a computation graph. The graph is a data structure that fully describes the computation you want to perform (e.g. training or inference of a machine learning model).

* That graph can be executed immediately, or stored for later.

* Since it's a serializable data structure, you can version it quite easily.

* You can deploy it to production without production having to depend on ANY of the code that built the graph, only the runtime necessary to execute it.

* You can run a compiler on it (such as XLA or TensorFlow's built in graph rewriter) to produce a more efficient version of the graph.

* In some circumstances, you can even compile the runtime away, producing a single .h/.o that you can link directly into e.g. a mobile app.

It's a beautiful and highly useful abstraction that allows TensorFlow to have both a great development and production story in one framework. Most frameworks only have a good story for either development or production.

If you are a machine learning researcher who doesn't need or care about deploying your work (i.e. mostly publishing papers), you may not want the overhead of having to deal with building a graph, and may prefer something that computes imperatively like PyTorch. If you are building products / services that use ML and developing/training your own models (as opposed to taking pre-trained models and using them), there is really no credible competitor to TensorFlow.

Disclaimer: I work at Google. I spend all day writing TensorFlow models. I'm not on the TensorFlow team nor do I speak for them or Google.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#36

Post by mmv »

k__ wrote: post #32

What's the best thing to build when starting TF? Like, the todo list of TF?

Tensorflow has a nice introduction tutorial using the MNIST dataset for recognition of handwritten digits.

Creating a small neural network and training it over the MNIST dataset is like the 'todo list' starter project of this kind of frameworks.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#37

Post by deepnotderp »

asah wrote: post #17

TensorFlow automatically discovers and uses GPUs and multiple cores, and I'm assuming Google is working on better supporting multiple GPUs, which currently requires hacks/tweaking to get speedups (it's easy to 'use' them) TensorFlow is a platform "winner" and approx 100% of all innovations will quickly be ported to TensorFlow - TBD which of the others will "keep up" with innovations as they continue to come out. othe…

Keras definitely means you lose control, but it's a tradeoff that's worth making in many cases.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#38

Post by digitalzombie »

sirfz wrote: post #24

We've moved over to Tensorflow from Theano around a year ago. I'm a Software Engineer on the team and here's what I think are advantages from my POV: 1) Transition was fairly straightforward, both APIs' interfaces are more-or-less similar and share some design characteristics. 2) Having said that, TF's API is easier to use and without a doubt a lot easier to read. 3) Consistency: Deploying Theano in different environ…

> 6) Its development is extremely rapid, new releases every couple of months with a lot of improvements and new features every time.

How stable is the api then?

I think google is a bit notorious for this (e.g. Angular vs Angular 2).

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#39

Post by asah »

laingc wrote: post #29
asah wrote: post #17

TensorFlow automatically discovers and uses GPUs and multiple cores, and I'm assuming Google is working on better supporting multiple GPUs, which currently requires hacks/tweaking to get speedups (it's easy to 'use' them) TensorFlow is a platform "winner" and approx 100% of all innovations will quickly be ported to TensorFlow - TBD which of the others will "keep up" with innovations as they continue to come out. othe…

Your comments are mostly valid, but I disagree about Keras. Although it's marvellous for patching something together quickly, if you want to branch out at all then it quickly becomes an absolute mess. Far better, in my view, is to work with the newer Estimators API. It is almost as fool-proof as Keras, but instead of trying to be a framework as such, the Estimators/learn API essentially just wraps up some of the boil…

I'll try it!

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#40

Post by asah »

anxman wrote: post #27

One other benefit with TensorFlow is that transitioning to cloud based processing on Google Cloud / Tensor Processing Units is seamless. It will turbo charge your training when compared to typical GPU performance. Disclosure: Work for Google Cloud

Any ETA on those TPUs? Stop teasing! ;-)
Post reply on HN