Growing open source from Torch to PyTorch • Hacker News Board

Re: Growing open source from Torch to PyTorch

#21

Post by smhx » Thu, Aug 05, 2021, 8:13 PM UTC

anonymoushn wrote: post #19

> So, over the years, I absorbed and appreciated that Torch was a user-centric product, which stood for immediate-mode, easy-to-debug, stay-out-of-the-way explicitness. It was targeted at people somewhat familiar with programming matters, and who could reason about things like performance, and if needed, write a C function and bind it in quickly. This paragraph sort of surprises me. In my experience if you want to do…

Whether using Lua or Python, for GPU-based scientific computing, the need to drop into a C call is the same. The overhead of Python vs Lua never really mattered.

While we were based on top of LuaJIT, we couldn't use the JIT for anything because we had to always call into the C library for GPU kernels (and LuaJIT can't JIT through an opaque C call, incase that C call changes the interpreter stack).

Where Python really helps is with its ecosystem. The entire data science and ML ecosystem is in Python.

The difficulty of calling out to C is not much greater in Python, things like PyBind11 make it pretty natural.

Re: Growing open source from Torch to PyTorch

#22

Post by anonymoushn » Fri, Aug 06, 2021, 5:05 AM UTC

smhx wrote: post #21

anonymoushn wrote: post #19

> So, over the years, I absorbed and appreciated that Torch was a user-centric product, which stood for immediate-mode, easy-to-debug, stay-out-of-the-way explicitness. It was targeted at people somewhat familiar with programming matters, and who could reason about things like performance, and if needed, write a C function and bind it in quickly. This paragraph sort of surprises me. In my experience if you want to do…

Whether using Lua or Python, for GPU-based scientific computing, the need to drop into a C call is the same. The overhead of Python vs Lua never really mattered. While we were based on top of LuaJIT, we couldn't use the JIT for anything because we had to always call into the C library for GPU kernels (and LuaJIT can't JIT through an opaque C call, incase that C call changes the interpreter stack). Where Python really…

People don't usually write the rules of a game or a MCTS to run on the GPU. They write it in Python, Lua, or C. If they write it in Python, then the GPU will idle all the time. If they write it in Lua or C then it will not.

Re: Growing open source from Torch to PyTorch

#23

Post by dklend122 » Fri, Aug 06, 2021, 1:00 PM UTC

snicker7 wrote: post #20

amkkma wrote: post #5

Earlier quoted context omitted.

>it has severe performance problems It had. It's now around parity with pytorch. And no, it wasn't about a usability tradeoff. It was about being more general- More general compiler, more general code, more composable code. Then, the team has been optimizing that and including compiler optimizations in the language that benefit all code. ML type code stressed that in a particular way. Pytorch does ML array heavy stuf…

> It had. It's now around parity with pytorch. In some cases, it is much faster. Consider Neural Stochastic Differential Equations, Flux is literally over 70,000x faster than Google's PyTorch-based implementation: https://gist.github.com/ChrisRackauckas/6a03e7b151c86b32d74b...

Yea, I meant parity for vanilla ML models. For anything off that beaten path it's much much faster

Re: Growing open source from Torch to PyTorch

#24

Post by dklend122 » Fri, Aug 06, 2021, 1:04 PM UTC

anonymoushn wrote: post #22

smhx wrote: post #21

Earlier quoted context omitted.

Whether using Lua or Python, for GPU-based scientific computing, the need to drop into a C call is the same. The overhead of Python vs Lua never really mattered. While we were based on top of LuaJIT, we couldn't use the JIT for anything because we had to always call into the C library for GPU kernels (and LuaJIT can't JIT through an opaque C call, incase that C call changes the interpreter stack). Where Python really…

People don't usually write the rules of a game or a MCTS to run on the GPU. They write it in Python, Lua, or C. If they write it in Python, then the GPU will idle all the time. If they write it in Lua or C then it will not.

In Julia it's easy to run the entire thing on CUDA: https://github.com/fabricerosay/AlphaGPU

No C or any other language required because Julia has GPU codegen.

You can also keep MCTS on the CPU and be competitive with cpp despite the code being higher level, easier to read and more generic and composable. See: https://github.com/jonathan-laurent/AlphaZero.jl