Is this the version combining Caffe2 and PyTorch into one framework? Is LMDB continuing as the default data load/store method?
PyTorch 1.0 is out
11–20 of 72 posts
Re: PyTorch 1.0 is out
#12What surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!)
[0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example
Re: PyTorch 1.0 is out
#13> The JIT is a set of compiler tools for bridging the gap between research in PyTorch and production. It allows for the creation of models that can run without a dependency on the Python interpreter and which can be optimized more aggressively. Using program annotations existing models can be transformed into Torch Script, a subset of Python that PyTorch can run directly. Isn't python bytecode simple enough that it c…
There was a huge problem with converting weight normalization module (torch.utils.weight_norm) and with forward and pre-forward hooks generally, so I had to rewrite the full model. Hope they've improved it in the stable release
Re: PyTorch 1.0 is out
#14TL;DR
- New JIT feature that lets you run your model without python. It now seems trivial to load a pytorch model in C++
- New distributed computation package. Major redesign.
- C++ frontend
- New torch hub feature to load models from github easily
Re: PyTorch 1.0 is out
#15> The JIT is a set of compiler tools for bridging the gap between research in PyTorch and production. It allows for the creation of models that can run without a dependency on the Python interpreter and which can be optimized more aggressively. Using program annotations existing models can be transformed into Torch Script, a subset of Python that PyTorch can run directly. Isn't python bytecode simple enough that it c…
Alternative implementations of Python don't seem very easy to write, so I'm guessing the answer is no.
Python is not very difficult to implement [1], the hard part is making it fast. That's because every property access involves a lot of magic behind the scenes, like __getattribute__, __getattr__ and the method resolution order. And because Python is dynamically typed, that dispatch logic can't be compiled away but needs to execute every time. (PyPy's JIT can probably speed it up, but still needs deoptimization checks in case types change.)
[1] see https://github.com/nedbat/byterun for a Python implementation
Re: PyTorch 1.0 is out
#16What surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!) [0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example
I'm hoping that this results in a nice, high-level API for https://github.com/fragcolor-xyz/nimtorch as well, which AFAIK has been wrapping the low-level Aten API. I've been keeping my eye on that project, and been really excited about it.
Re: PyTorch 1.0 is out
#17Is this the version combining Caffe2 and PyTorch into one framework? Is LMDB continuing as the default data load/store method?
According to this [0] blog, it it:
PyTorch 1.0 takes the modular, production-oriented capabilities from Caffe2 and ONNX and combines them with PyTorch's existing flexible, research-focused design to provide a fast, seamless path from research prototyping to production deployment for a broad range of AI projects.
[0] https://developers.facebook.com/blog/post/2018/05/02/announc...
Re: PyTorch 1.0 is out
#18What surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!) [0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example
i picked up c++ for some gpu stuff with the arrayfire api... felt the same. firstly, modern c++ takes no time to learn if you come from java / c/ c# / etc. secondly, things like operator overloading and type inference make for pretty seamless apis. E.g. want to add matrices? auto C = A + B.
A lot of things suck (closures, generator functions, first order functions all suck in c++), but oh my does it all run fast when you get it working; plus wrapping it with Rcpp or Lua is easy as pie.
Re: PyTorch 1.0 is out
#19In case anybody else was wondering, since this isn't in the fine article:
"PyTorch is a Python package that provides two high-level features:
* Tensor computation (like NumPy) with strong GPU acceleration
* Deep neural networks built on a tape-based autograd system
You can reuse your favorite Python packages such as NumPy, SciPy and Cython to extend PyTorch when needed."
Re: PyTorch 1.0 is out
#20What surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!) [0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example
Wow, this is cool. How complete is the API? Could you use it for research? Last I checked, the TensorFlow C++ API was missing all sorts of important stuff for building models and was basically only useful for loading models saved from Python.