Sill no support for/from Apple Silicon?
[deleted]
PyTorch 2.0
21–30 of 111 posts
Re: PyTorch 2.0
#22Sill no support for/from Apple Silicon?
It's supported since 1.12[0], no? [0]: https://pytorch.org/blog/introducing-accelerated-pytorch-tra...
It's also officially in beta as of 1.13 https://pytorch.org/blog/PyTorch-1.13-release/#beta-support-...
Re: PyTorch 2.0
#23Earlier quoted context omitted.
Not sure I understand that question, is versioning the biggest problem no, but it costs nothing to keep semver and prevent production headaches later. If you meant inference speed then yeah it's a very big problem so it's good that they are addressing it.
what exact production headaches you are expecting by bump the number from 1.13 -> 2.0, while all existing codes keep working as before? And how is it different from bumping 1.13 to 1.14, even if they named it 1.14?
The soft kind. Major versions are deeply ingrained as "possible backward-compatibility issues" in most engineers' brain. If you handle model development, evaluation and deployment yourself than sure you won't have any issues, but in a bigger organization you have to get people to switch and that version number will mean that everyone will ask the same "hang on this is a major version change?!" question every step of the way.
Re: PyTorch 2.0
#24Re: PyTorch 2.0
#25A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect.
Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects. Meta reportedly had more than 300 (?) people working on PyTorch, helping the community resolve issues, producing tons of high-quality documentation and libraries, and marketing themselves nicely in all kinds of conferences and media.
Re: PyTorch 2.0
#26Earlier quoted context omitted.
PyTorch and JAX are both open-source libraries for developing machine learning models, but they have some important differences. PyTorch is a more general-purpose library that provides a wide range of functionalities for developing and training machine learning models. It also has strong support for deep learning and is used by many researchers and companies in production environments. JAX, on the other hand, is desi…
I was reading this and thinking it was a pretty terrible answer - glad it is just generated by an AI and not you personally so I'm not insulting you. JAX is basically numpy on steroids and lets you do a lot of non-standard things (like a differentiable physics simulation or something) that would be harder with Pytorch. They are both "high-performance." Pytorch is more geared towards traditional deep learning and has…
jax is not numpy on steroids. jax is "use python idiomatically to generate optimized XLA code for evaluating functions both forward and backward."
Re: PyTorch 2.0
#27Earlier quoted context omitted.
I was reading this and thinking it was a pretty terrible answer - glad it is just generated by an AI and not you personally so I'm not insulting you. JAX is basically numpy on steroids and lets you do a lot of non-standard things (like a differentiable physics simulation or something) that would be harder with Pytorch. They are both "high-performance." Pytorch is more geared towards traditional deep learning and has…
jax is not numpy on steroids. jax is "use python idiomatically to generate optimized XLA code for evaluating functions both forward and backward."
Probably the primary use of jax is `jax.numpy` which is XLA accelerated and differentiable numpy.
I'll admit that saying "basically numpy on steroids" might have been an overreduction. It is a system for function transformations that is built on XLA and oriented towards science & ML applications.
It's not just me saying stuff like this.
François Chollet (creator of Keras): "[jax is] basically Numpy with gradients. And it can compile to XLA, for strong GPU/TPU acceleration. It's an ideal fit for researchers who want maximum flexibility when implementing new ideas from scratch."
Re: PyTorch 2.0
#28> We believe that this is a substantial new direction for PyTorch – hence we call it 2.0. torch.compile is a fully additive (and optional) feature and hence 2.0 is 100% backward compatible by definition. How about just calling it PyTorch 1.14 if it's backward compatible? Version numbering shouldn't be used as a marketing gimmick.
Dismissive comments like this make me not want to read HN anymore and in addition it’s against the HN guidelines:
It’s snarky. It’s incurious. It’s neither thoughtful nor substantive. It’s flame bait. It’s a shallow dismissal. It doesn’t teach anything. It’s the most provocative thing to complain about.
https://news.ycombinator.com/newsguidelines.html
I’m sorry I had to leave this comment, so let me also try to respond thoughtfully:
Assuming that PyTorch is using semantic versioning requires that the major version MUST change when making a backwards incompatible API change:
> Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented.
This requirement does NOT preclude changing the major version when making backwards-compatible changes.
PyTorch has not violated semver here. It is absolutely compatible with semver to bump the major version for marketing reasons.
Re: PyTorch 2.0
#29> Today, we announce torch.compile, a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python. I'll admit I don't know enough about PyTorch to know what torch.compile is exactly. But does this means some features of PyTorch will no longer be available in the core C++ library? One of the nice things about PyTorch had been that you could do your training…
One of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application.
Or even train in C++ or Rust without much loss in functionality.
Re: PyTorch 2.0
#30> Today, we announce torch.compile, a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python. I'll admit I don't know enough about PyTorch to know what torch.compile is exactly. But does this means some features of PyTorch will no longer be available in the core C++ library? One of the nice things about PyTorch had been that you could do your training…
Facebook did similar thing to its original code PHP, it uses HHVM to 'compile' PHP(now called Hacklang) to gain performance, it seems doing similar thing with python here.