Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why TensorFlow instead of Theano for deep learning?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#11

Post by chenzhekl »

DLEnthusiast wrote: post #10
chenzhekl wrote: post #9

Earlier quoted context omitted.

Few people use PyTorch largely because it is relatively new (0.1.12). It even doesn't have distributed training capabilities (coming in 0.2). Your arguments don't say anything about frameworks themselves. It is unfair! When people say PyTorch is better for research, they mean it is more flexible, and it is easier to implement non-trivial network architectures with it, such as recursive network, which is a cumbersome…

All of these frameworks are "relatively new". TensorFlow: 1.6 years. CNTK: 1 year. PyTorch: 0.5 year. Are they really impossible to compare? > When people say PyTorch is better for research, they mean That's not what "people" say. They tend to say the opposite. Maybe we can ask OP what he meant when he said it. > it is easier to implement non-trivial network architectures with it, such as recursive network It is inte…

> All of these frameworks are "relatively new". TensorFlow: 1.6 years. CNTK: 1 year. PyTorch: 0.5 year. Are they really impossible to compare?

1.6 years is a long time in DL community.

> That's not what "people" say. They tend to say the opposite. Maybe we can ask OP what he meant when he said it.

Go ahead! Ask it.

> They use Chainer and DyNet.

You know Chainer came far before PyTorch, which heavily influenced PyTorch's design. You are always saying XX is using XX. Why not talk about frameworks themselves?

If you insisted to your idea, let's settle down. I don't want to start a framework war. I just want all frameworks to be equally considered.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#12

Post by sweezyjeezy »

DLEnthusiast wrote: post #7
visarga wrote: post #4

TensorFlow is better for deployment. Pytorch is better for research. Theano/Keras is simpler to use and a little faster than TensorFlow

"PyTorch is better for research" is a weird, unsubstantiated statement. The fact is that few serious researchers use PyTorch (and even those complain about it). It's mostly grad students in a handful of labs. The only researchers I know who use PyTorch have been from FaceBook, and that's because they were implicitly forced to use it (PyTorch is developed by FaceBook). According to https://medium.com/@karpathy/icml-ac…

Your list of who uses what is kind of contaminated by who wrote the code, I don't think it proves anything.

Obviously Google Brain use TF and Montreal use Theano - they wrote them. Deepmind use TF, but they used Torch before the google takeover. Similarly Google and Toronto are deeply intertwined.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#13

Post by cs702 »

The main reason to bet on TensorFlow is that it seems to have by far the greatest adoption of all frameworks, as evidenced by github statistics, HN polls, and other surveys:

* https://twitter.com/fchollet/status/765212287531495424

* https://news.ycombinator.com/item?id=12391744

* https://github.com/aymericdamien/TopDeepLearning

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#14

Post by paulsutter »

Fortunately, it's not an irrevocable decision like choosing a JavaScript framework. With deep learning you spend a lot of time considering a small amount of code.

We use several frameworks because sample code from different papers uses different frameworks. It's not that big of a deal.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#15

Post by ssivark »

An observation when taking a step back: The discussion about deep learning frameworks seems almost as complicated as the Javascript framework discussions a couple of years ago. Google and Facebook pushing their own frameworks (among other participants) also adds to the deja vu!

Why is the choice of framework such a big deal? Is it unreasonable to expect someone well-versed in one framework to be able to pick up another reasonably fast if/when collaborating with someone proficient in the latter ?

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#17

Post by asah »

TensorFlow automatically discovers and uses GPUs and multiple cores, and I'm assuming Google is working on better supporting multiple GPUs, which currently requires hacks/tweaking to get speedups (it's easy to 'use' them)

TensorFlow is a platform "winner" and approx 100% of all innovations will quickly be ported to TensorFlow - TBD which of the others will "keep up" with innovations as they continue to come out.

other recommendations:

- by default, TensorFlow allocates 100% of GPU RAM for each process. You'll want to control this: https://stackoverflow.com/questions/34199233/how-to-prevent-...

- Keras. yes, this. Dramatically reduces code by 2-10x, without loss of control AFAICT.

- cloud hardware. Pretty quickly, you'll want to scale and run multiple tests at once, and e.g. quickly backup & copy data, replicate system images, etc. I use Google Cloud Hosting and it's much easier (and cheaper) than AWS. Haven't tried Azure but heard good things. At least once, Google's internet bandwidth has saved hours waiting for data transfers.

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#18

Post by ma2rten »

Here are some reasons why TensorFlow might be better:

* more widely used, more example code

* developed by a bigger team, likely to improve faster

* easier to deploy

* training with Cloud ML

* better support for distributed training

* no compile time (this can be long especially for RNNs)

Re: Ask HN: Why TensorFlow instead of Theano for deep learning?

#19

Post by pilooch »

DLEnthusiast wrote: post #10
chenzhekl wrote: post #9

Earlier quoted context omitted.

Few people use PyTorch largely because it is relatively new (0.1.12). It even doesn't have distributed training capabilities (coming in 0.2). Your arguments don't say anything about frameworks themselves. It is unfair! When people say PyTorch is better for research, they mean it is more flexible, and it is easier to implement non-trivial network architectures with it, such as recursive network, which is a cumbersome…

All of these frameworks are "relatively new". TensorFlow: 1.6 years. CNTK: 1 year. PyTorch: 0.5 year. Are they really impossible to compare? > When people say PyTorch is better for research, they mean That's not what "people" say. They tend to say the opposite. Maybe we can ask OP what he meant when he said it. > it is easier to implement non-trivial network architectures with it, such as recursive network It is inte…

With all due respect you don't appear to know much about what you are commenting about. There is a huge community of research and applications around rnn and all other architectures. Pytorch has an extremely vivid and fast growing codebase, across all neural architectures and applications, it's remarkable actually. One reason might be because it is simple an effective, including easy debugging. Another is because research now is much focused on new possibly complex and exotic architectures, new optimizers, inner guts and behavior understanding​, including theoretically. And it appears pytorch makes that easy. Don't throw it away too fast as a DL enthusiast :)
Post reply on HN