Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch – Tensors and Dynamic neural networks in Python

pytorch.org

Post reply on HN
Print view View original

61–70 of 91 posts

Re: PyTorch – Tensors and Dynamic neural networks in Python

#61

Post by superfx »

Smerity wrote: post #57
superfx wrote: post #55

Earlier quoted context omitted.

How is adding dynamic graphs to TensorFlow "after the fact" while adding it to Torch isn't? (Torch is much older than TF).

Torch was never written as a static graph computation framework. Torch was/is more a tensor manipulation library where you are executing the individual operations step by step and the graph can be tracked and constructed incrementally from those operations. For this reason, much of PyTorch is about building a layer on top of the underlying components (which are focused on efficiently manipulating tensors and/or imple…

Thanks for the clarification.

Does the upcoming XLA interact with this as well? I.e. compilation would be too costly for dynamic graphs, and so it would only make sense for static graphs?

Re: PyTorch – Tensors and Dynamic neural networks in Python

#62

Post by pavanky »

zo7 wrote: post #11
aaron-lebo wrote: post #5

Is this related to lua's Torch at all? http://torch.ch/

They don't seem to explicitly say it, but it might be using the same core code given the structure of the framework and their mentioning that it's a mature codebase several years old. The license file also goes back to NYU before being taken over by Facebook, similar to Torch.

They are sharing the code base using git-subtrees. So the C and CUDA parts of the codebase will be kept in sync. The modules written in lua or python will diverge.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#63

Post by pavanky »

aaron-lebo wrote: post #5

Is this related to lua's Torch at all? http://torch.ch/

They share the same underlying C and CUDA libraries. The Python and Lua modules are different. You can see both projects have pretty much the same contributors because they are sharing the code base using git-subtree.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#64

Post by Smerity »

superfx wrote: post #61
Smerity wrote: post #57

Earlier quoted context omitted.

Torch was never written as a static graph computation framework. Torch was/is more a tensor manipulation library where you are executing the individual operations step by step and the graph can be tracked and constructed incrementally from those operations. For this reason, much of PyTorch is about building a layer on top of the underlying components (which are focused on efficiently manipulating tensors and/or imple…

Thanks for the clarification. Does the upcoming XLA interact with this as well? I.e. compilation would be too costly for dynamic graphs, and so it would only make sense for static graphs?

I am not highly clued in to XLA as it's new, quite experimental, and most honestly I've just not looked at it in detail. Given XLA provides compilation, JIT or ahead of time, it doesn't really (yet) factor in to the dynamic graph discussion.

What would theoretically be interesting is a JIT for dynamic computation graphs. Frequent subgraphs could be optimized and cached and re-used when appropriate, similar to a JIT for Javascript. No doubt they're already pondering such things.

https://www.tensorflow.org/versions/master/experimental/xla/

Re: PyTorch – Tensors and Dynamic neural networks in Python

#65

Post by attractivechaos »

wodenokoto wrote: post #32
vegabook wrote: post #25

Guess there's no escaping Python. I had hoped Lua(jit) might emerge as a scientific programming alternative but with Torch now throwing its hat into the Python ring I sense a monoculture in the making. Bit of a shame really because Lua is a nice language and was an interesting alternative.

There's also R and Julia and there are still plenty of people building neural networks in C.

It is easy to build a multi-layer perceptron purely in C. You can roll your own or use a library like FANN. However, so far as I know, very few (darknet is the only example I know of) are using C to build a bit more complex networks like CNN/RNN, let alone those topologically complex networks in research domain.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#66

Post by eudoxus »

gallerdude wrote: post #52

What's the highest level neural network lib I can use? I'm a total programming idiot but I find neural nets fascinating.

This[1] was posted earlier today to HN. Seems pretty simple to play with NNs without coding.

[1] - http://kur.deepgram.com/

Re: PyTorch – Tensors and Dynamic neural networks in Python

#69

Post by chewxy »

Smerity wrote: post #33

Only a few months ago people saying that the deep learning library ecosystem was starting to stabilize. I never saw that as the case. The latest frontier for deep learning libraries is ensuring efficient support for dynamic computation graphs. Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we're processing text, one example being a few words while anot…

If you guys wanna use Go, Gorgonia also features dynamic graphs the way Chainer does (also Theano-style compile-execute machines)

Re: PyTorch – Tensors and Dynamic neural networks in Python

#70

Post by apaszke »

visarga wrote: post #67
gallerdude wrote: post #52

What's the highest level neural network lib I can use? I'm a total programming idiot but I find neural nets fascinating.

Keras requires just a few lines of code, it's designed for easy use and practicality.

torch.nn offers a very similar interface to Keras (e.g. see Alexnet definition at https://github.com/pytorch/vision/blob/master/torchvision/mo...).
Post reply on HN