Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch 2.0

pytorch.org

Post reply on HN
Print view View original

61–70 of 111 posts

Re: PyTorch 2.0

#61

Post by not2b »

wging wrote: post #57
hintymad wrote: post #25

A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect. Anoth…

> Meta reportedly had more than 300 (?) people working on PyTorch, How much did this change after the big Meta layoffs? I think I know people who are no longer there, but I haven't talked to them about it yet.

Because Meta open sourced it, and because PyTorch caught on, hopefully some of those laid off people can continue to work on it and also market themselves as PyTorch experts.

Re: PyTorch 2.0

#63

Post by microtonal »

synergy20 wrote: post #31
microtonal wrote: post #29

Earlier quoted context omitted.

One of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application. Or even train in C++ or Rust without much loss in functionality.

Rust really has not had any presence in AI training engine yet, it's probably 100% c++.

I was referring to the libtorch library, which you can use through the tch crate. It is possible to make such rich bindings because so much of Torch is exposed through the C++ API. When more new functionality is moved to Python, it makes it harder to use functionality from the C++ interface and downstream bindings.

Re: PyTorch 2.0

#64

Post by adgjlsfhk1 »

patrickkidger wrote: post #51
uoaei wrote: post #32

Earlier quoted context omitted.

Can someone comment more on what makes JAX that much better for differentiable simulations than PyTorch? I'm working on a new module for work and none of my colleagues have much experience developing ML per se. I'm trying to decide whether to force their hand by implementing v1 in PyTorch or JAX and differentiable physics simulations is a likely future use case. Why is PyTorch harder?

At least prior to this announcement: JAX was much faster than PyTorch for differentiable physics. (Better JIT compiler; reduced Python-level overhead.) E.g for numerical ODE simulation, I've found that Diffrax ( https://github.com/patrick-kidger/diffrax ) is ~100 times faster than torchdiffeq on the forward pass. The backward pass is much closer, and for this Diffrax is about 1.5 times faster. It remains to be seen h…

If you care about performance of differential physics you shouldn't use python. Diffrax is almost OKish, but is missing a ton of features (e.g. good stiff solvers, arbitrary precision support, events for anything other than stopping the simulation, ability to control the linear solve which are needed for large problems). For simple cases it can come close to the C++/Julia solvers, but for anything complicated, you either won't be able to formulate the model, or you won't be able to solve it efficiently.

Re: PyTorch 2.0

#65

Post by PartiallyTyped »

wging wrote: post #57
hintymad wrote: post #25

A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect. Anoth…

> Meta reportedly had more than 300 (?) people working on PyTorch, How much did this change after the big Meta layoffs? I think I know people who are no longer there, but I haven't talked to them about it yet.

NB: PyTorch is now under the Linux Foundation.

https://news.ycombinator.com/item?id=32810976

Re: PyTorch 2.0

#66

Post by PartiallyTyped »

amelius wrote: post #40
ansk wrote: post #39

So this looks like a further convergence of the tensorflow and pytorch APIs (the lower-level APIs at least). Tensorflow was designed with compilable graphs as the primary execution model and as part of their 2.0 release, they redesigned the APIs to encompass eager execution as well. Pytorch is coming from the other end, with eager execution being the default and now emphasizing improved tools for graph compilation in…

How much performance can be squeezed from going from the plain python API to the graph-based solution, typically?

It depends.. Using Jax to compile down to XLA, I often saw >2 orders of magnitude improvements. This however was roughly 6 months ago.

Re: PyTorch 2.0

#67

Post by PartiallyTyped »

chazeon wrote: post #12

How is PyTorch compares to JAX and its stack?

I find that Jax tends to result in messy code unless you build good abstractions. I personally don't like Flax and Haiku, I prefer stax and Equinox as they are more transparent on what is happening, feel a lot less like magic, and more pythonic (explicit is better than implicit etc).

PyTorch is far more friendly for deep learning stuff, but sometimes all you want is pure numerical computations that can be vmapped across tensors, and this is where jax shines imho.

Personal Example: I needed to sample a bunch of datapoints, make distributions out of them, sample, and then compute the density of each sample across distributions. Doing this with pytorch was rather slow, I was probably doing something wrong with vectorization and broadcasting, but I didn't have the time to figure it out.

With jax, I wrote a function that produces the samples, then I vmapped the evaluation of a sample across all distributions, then vmapped over all samples. Took a couple of minutes to implement and seconds to execute.

PyTorch also has the advantage of a far more mature ecosystem, libraries like Lightning, Accelerate, Transformers, Evaluate, and so on make building models a breeze.

Re: PyTorch 2.0

#68

Post by fddr »

quietbritishjim wrote: post #5

> Today, we announce torch.compile, a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python. I'll admit I don't know enough about PyTorch to know what torch.compile is exactly. But does this means some features of PyTorch will no longer be available in the core C++ library? One of the nice things about PyTorch had been that you could do your training…

The `torch.compile` API itself will not be available from C++. That means that you won't get the pytorch 2.0 performance gains if you use it via C++ API.

There's no plan to deprecate the existing C++ API, it should keep working as it is. However, a common theme of all the changes is implementing more of pytorch in python (explicitly the goal of primtorch), so if this plan works it could happen in the long run.

Re: PyTorch 2.0

#69

Post by levesque »

la_fayette wrote: post #53
SleekEagle wrote: post #45

Earlier quoted context omitted.

Exactly, especially in the age of ridiculously rapid development that we have found ourselves in over the past few years. This is exactly why TensorFlow is dying

:) it still is impossible to bring a cnn rnn network in pytorch to mobile, which works fine with tflite...

I don't think that's a problem for the vast majority of pytorch users

Re: PyTorch 2.0

#70

Post by vikinghckr »

hintymad wrote: post #25

A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect. Anoth…

What I really find interesting here is that PyTorch, a library maintained by Facebook, is winning the marketshare and mindshare due to clean API, whereas Tensorflow, maintained by Google, is losing due to inferior API. In general, Google as a company emphasizes code quality and best practices far more than Facebook. But the story was reversed here.
Post reply on HN