Growing open source from Torch to PyTorch • Hacker News Board

Re: Growing open source from Torch to PyTorch

#11

Post by mirker » Wed, Aug 04, 2021, 9:27 PM UTC

smhx wrote: post #3

posharma wrote: post #2

PyTorch is amazing. The article was a good read. Although I'm confused. How can a ML framework be not obsessed with speed/performance?

Author here. Being conscious about speed and performance is different from making that your competitive advantage or USP. Our main focus is usability, and one of our secondary focuses is to not look like clowns in the performance department. So, we try to take more decisions that trade off performance for usability than vice versa.

Thanks for the post.

One question: One of the advantages about having a clean design is that performance is easier to optimize, since the 80%/20% rule of performance becomes much more obvious. How true was this in your experience? Were there any major performance-related design changes or was performance optimization a matter of tuning a few selected functions?

Re: Growing open source from Torch to PyTorch

#12

Post by amkkma » Wed, Aug 04, 2021, 9:28 PM UTC

jsinai wrote: post #9

amkkma wrote: post #5

Earlier quoted context omitted.

>it has severe performance problems It had. It's now around parity with pytorch. And no, it wasn't about a usability tradeoff. It was about being more general- More general compiler, more general code, more composable code. Then, the team has been optimizing that and including compiler optimizations in the language that benefit all code. ML type code stressed that in a particular way. Pytorch does ML array heavy stuf…

> In the meantime, places like MIT, moderna, NASA etc are reaping the benefits. Can you elaborate more? MIT is well known but would interesting to know how Moderna and NASA are using Flux?

Sure!

NASA: https://www.youtube.com/watch?v=tQpqsmwlfY0

Moderna: https://pumas.ai/ https://discourse.julialang.org/t/has-moderna-used-pumas-ai-...

There are many many more. These unique and sought after capability are what got Julia Computing its 24 mil series A (https://twitter.com/Viral_B_Shah/status/1417128416206376960)

Re: Growing open source from Torch to PyTorch

#13

Post by amkkma » Wed, Aug 04, 2021, 9:32 PM UTC

smhx wrote: post #6

ampdepolymerase wrote: post #4

Earlier quoted context omitted.

You are doing a good job balancing the two. For Julia's Flux, they did the opposite and it has severe performance problems compared to PyTorch despite being more usable and easier to install. Installing PyTorch with Poetry is next to impossible. Flux got this right by bundling the GPU drivers. Their installation is also standardized and does not require the weird pip -f flag for CPU only installations.

We ship everything needed for userland -- including parts of CUDA/CuBLAS and CuDNN that we need (which is why our binaries are so fat). GPU drivers would be kernel-land and I don't think we actually can install GPU drivers as part of a `pip install`. Will look into what Flux is doing, but I doubt they ship GPU drivers. Separately, thanks for flagging the Poetry issue, we might prioritize it, especially if the fix is…

yes Flux doesn't ship GPU drivers. It ships everything else (like CUDA toolkit etc) as needed, using the artifact / pkg system, for all mainstream OSes. Doesn't interfere with system libraries.

https://julialang.org/blog/2019/11/artifacts/

Re: Growing open source from Torch to PyTorch

#14

Post by amkkma » Wed, Aug 04, 2021, 9:34 PM UTC

As a julia user, thanks for this! Inspiring and packed with pearls. There's a lot we can learn from the python community

Re: Growing open source from Torch to PyTorch

#15

Post by jimsimmons » Wed, Aug 04, 2021, 10:21 PM UTC

albertzeyer wrote: post #7

It was necessary to move away from Lua to stay relevant within the machine learning community. Python was a natural choice because there was Theano and TensorFlow. PyTorch could make use of the best API ideas from the other frameworks (also higher-level like Keras). And it was executed well. All these core principles of easy debuggability are indeed very important to win developers. Clean code, understandable code, f…

Keras was a copy of Torch API. If you read the original Keras readme it literally says so.

Re: Growing open source from Torch to PyTorch

#16

Post by geofft » Thu, Aug 05, 2021, 1:01 AM UTC

smhx wrote: post #6

ampdepolymerase wrote: post #4

Earlier quoted context omitted.

You are doing a good job balancing the two. For Julia's Flux, they did the opposite and it has severe performance problems compared to PyTorch despite being more usable and easier to install. Installing PyTorch with Poetry is next to impossible. Flux got this right by bundling the GPU drivers. Their installation is also standardized and does not require the weird pip -f flag for CPU only installations.

We ship everything needed for userland -- including parts of CUDA/CuBLAS and CuDNN that we need (which is why our binaries are so fat). GPU drivers would be kernel-land and I don't think we actually can install GPU drivers as part of a `pip install`. Will look into what Flux is doing, but I doubt they ship GPU drivers. Separately, thanks for flagging the Poetry issue, we might prioritize it, especially if the fix is…

You might want to take a look at https://discuss.python.org/t/what-to-do-about-gpus-and-the-b... if you haven't seen it - there's a practical problem that hosting built GPU code on PyPI is very difficult.

Re: Growing open source from Torch to PyTorch

#17

Post by gspr » Thu, Aug 05, 2021, 7:00 AM UTC

smhx wrote: post #6

ampdepolymerase wrote: post #4

Earlier quoted context omitted.

You are doing a good job balancing the two. For Julia's Flux, they did the opposite and it has severe performance problems compared to PyTorch despite being more usable and easier to install. Installing PyTorch with Poetry is next to impossible. Flux got this right by bundling the GPU drivers. Their installation is also standardized and does not require the weird pip -f flag for CPU only installations.

We ship everything needed for userland -- including parts of CUDA/CuBLAS and CuDNN that we need (which is why our binaries are so fat). GPU drivers would be kernel-land and I don't think we actually can install GPU drivers as part of a `pip install`. Will look into what Flux is doing, but I doubt they ship GPU drivers. Separately, thanks for flagging the Poetry issue, we might prioritize it, especially if the fix is…

I'll hijack your presence and a post about CUDA/cuBLAS: is there any news about OpenCL support?

Re: Growing open source from Torch to PyTorch

#18

Post by enriquto » Thu, Aug 05, 2021, 7:37 AM UTC

albertzeyer wrote: post #7

It was necessary to move away from Lua to stay relevant within the machine learning community. Python was a natural choice because there was Theano and TensorFlow. PyTorch could make use of the best API ideas from the other frameworks (also higher-level like Keras). And it was executed well. All these core principles of easy debuggability are indeed very important to win developers. Clean code, understandable code, f…

> It was necessary to move away from Lua

Why? I sort of became disillusioned in torch after they abandoned lua.

Re: Growing open source from Torch to PyTorch

#19

Post by anonymoushn » Thu, Aug 05, 2021, 9:38 AM UTC

> So, over the years, I absorbed and appreciated that Torch was a user-centric product, which stood for immediate-mode, easy-to-debug, stay-out-of-the-way explicitness. It was targeted at people somewhat familiar with programming matters, and who could reason about things like performance, and if needed, write a C function and bind it in quickly.

This paragraph sort of surprises me. In my experience if you want to do anything other than calling out to numeric libraries, you can do it in Lua and it will work, or you can do it in Python and suddenly your machine learning pipeline will spend 95% of its time running Python while your GPU idles. So the need to be able to drop down to C is much more severe in Python, and the difficulty of calling out to C is much greater.

Re: Growing open source from Torch to PyTorch

#20

Post by snicker7 » Thu, Aug 05, 2021, 7:42 PM UTC

amkkma wrote: post #5

ampdepolymerase wrote: post #4

Earlier quoted context omitted.

You are doing a good job balancing the two. For Julia's Flux, they did the opposite and it has severe performance problems compared to PyTorch despite being more usable and easier to install. Installing PyTorch with Poetry is next to impossible. Flux got this right by bundling the GPU drivers. Their installation is also standardized and does not require the weird pip -f flag for CPU only installations.

>it has severe performance problems It had. It's now around parity with pytorch. And no, it wasn't about a usability tradeoff. It was about being more general- More general compiler, more general code, more composable code. Then, the team has been optimizing that and including compiler optimizations in the language that benefit all code. ML type code stressed that in a particular way. Pytorch does ML array heavy stuf…

> It had. It's now around parity with pytorch.

In some cases, it is much faster.

Consider Neural Stochastic Differential Equations, Flux is literally over 70,000x faster than Google's PyTorch-based implementation:

https://gist.github.com/ChrisRackauckas/6a03e7b151c86b32d74b...