Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch – Tensors and Dynamic neural networks in Python

pytorch.org

Post reply on HN
Print view View original

31–40 of 91 posts

Re: PyTorch – Tensors and Dynamic neural networks in Python

#31

Post by jjawssd »

vegabook wrote: post #30
jjawssd wrote: post #29

Earlier quoted context omitted.

Lua is extremely flexible to the point where there is basically no standard library. This causes problems with code reuse and moving between codebases because everyone does things drastically differently. Compare this to Numpy in the Python world, a single fundamental package for scientific computing in Python. Lua is less used than Python in the scientific community, and a lot of the most innovative machine learning…

Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disc…

A very large portion of performance problems can be mitigated with the use of cython and the new asyncio stuff.

asyncio success story: https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho...

cython: http://scikit-learn.org/stable/developers/performance.html

Re: PyTorch – Tensors and Dynamic neural networks in Python

#32

Post by wodenokoto »

vegabook wrote: post #25

Guess there's no escaping Python. I had hoped Lua(jit) might emerge as a scientific programming alternative but with Torch now throwing its hat into the Python ring I sense a monoculture in the making. Bit of a shame really because Lua is a nice language and was an interesting alternative.

There's also R and Julia and there are still plenty of people building neural networks in C.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#33

Post by Smerity »

Only a few months ago people saying that the deep learning library ecosystem was starting to stabilize. I never saw that as the case. The latest frontier for deep learning libraries is ensuring efficient support for dynamic computation graphs.

Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we're processing text, one example being a few words while another being paragraphs of text, or when we are performing operations against a tree structure of variable size. This problem is particularly prominent in particular subfields, such as natural language processing, where I spend most of my time.

PyTorch tackles this very well, as do Chainer[1] and DyNet[2]. Indeed, PyTorch construction was directly informed from Chainer[3], though re-architected and designed to be even faster still. I have seen all of these receive renewed interest in recent months, particularly amongst many researchers performing cutting edge research in the domain. When you're working with new architectures, you want the most flexibility possible, and these frameworks allow for that.

As a counterpoint, TensorFlow does not handle these dynamic graph cases well at all. There are some primitive dynamic constructs but they're not flexible and usually quite limiting. In the near future there are plans to allow TensorFlow to become more dynamic, but adding it in after the fact is going to be a challenge, especially to do efficiently.

Disclosure: My team at Salesforce Research use Chainer extensively and my colleague James Bradbury was a contributor to PyTorch whilst it was in stealth mode. We're planning to transition from Chainer to PyTorch for future work.

[1]: http://chainer.org/

[2]: https://github.com/clab/dynet

[3]: https://twitter.com/jekbradbury/status/821786330459836416

Re: PyTorch – Tensors and Dynamic neural networks in Python

#34

Post by vegabook »

jjawssd wrote: post #31
vegabook wrote: post #30

Earlier quoted context omitted.

Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disc…

A very large portion of performance problems can be mitigated with the use of cython and the new asyncio stuff. asyncio success story: https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho... cython: http://scikit-learn.org/stable/developers/performance.html

Luajit is at least 10x faster than python and easily obviates the need to mess around with cython. That's an easy win for Lua. Let's be honest: Torch has decided that if you cannot beat them, join them. It is about network effects. Not about Python better than Lua intrinsically.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#35

Post by cr0sh »

vegabook wrote: post #30
jjawssd wrote: post #29

Earlier quoted context omitted.

Lua is extremely flexible to the point where there is basically no standard library. This causes problems with code reuse and moving between codebases because everyone does things drastically differently. Compare this to Numpy in the Python world, a single fundamental package for scientific computing in Python. Lua is less used than Python in the scientific community, and a lot of the most innovative machine learning…

Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disc…

> And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data.

Then use Lua for that, if you are more comfortable there and want/need the speed bump. There's nothing that says an entire project or whatnot has to be developed in a singular language.

Use each tool to its strengths, as your needs, requirements, and abilities dictate.

Re: PyTorch – Tensors and Dynamic neural networks in Python

#36

Post by jjawssd »

vegabook wrote: post #34
jjawssd wrote: post #31

Earlier quoted context omitted.

A very large portion of performance problems can be mitigated with the use of cython and the new asyncio stuff. asyncio success story: https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho... cython: http://scikit-learn.org/stable/developers/performance.html

Luajit is at least 10x faster than python and easily obviates the need to mess around with cython. That's an easy win for Lua. Let's be honest: Torch has decided that if you cannot beat them, join them. It is about network effects. Not about Python better than Lua intrinsically.

Can't argue with that

Re: PyTorch – Tensors and Dynamic neural networks in Python

#38

Post by vegabook »

cr0sh wrote: post #35
vegabook wrote: post #30

Earlier quoted context omitted.

Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disc…

> And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Then use Lua for that, if you are more comfortable there and want/need the speed bump. There's nothing that says an entire project or whatnot has to be developed in a singular language. Use each tool to its strengths, as your needs, requirements, and abilities dictate.

There always has to be someone rolling out the horses-for-courses pitch. No. I wanted Lua to gain traction with other people. That's the point. I would have liked the Lua sci-ecosystem to be healthy as an alternative.
Post reply on HN