91–100 of 113 posts
PyTorch is the javascript of ML. sadly "worse is better" software has better survival characteristics even when there is consensus that technology X is theoretically better
How is Jax theoretically better than PyTorch? The author is ignorant of torch.compile and biased as other commenters have pointed out.
PyTorch is developed by multiple companies / stake holders while jax is google only with internal tooling they don’t share with the world. This alone is a major reason not to use jax. Also I think it is more the other way around: with torch.compile the main advantage of jax is disappearing.
> with torch.compile the main advantage of jax is disappearing. Interesting take - I agree here somewhat. But also, wouldn't you think a framework that has been from the ground-up designed around a specific, mature compiler stack be better able to integrate compilers in a more stable fashion than just shoe-horning static compilers into a very dynamic framework? ;)
PyTorch is developed by multiple companies / stake holders while jax is google only with internal tooling they don’t share with the world. This alone is a major reason not to use jax. Also I think it is more the other way around: with torch.compile the main advantage of jax is disappearing.
> with torch.compile the main advantage of jax is disappearing. Interesting take - I agree here somewhat. But also, wouldn't you think a framework that has been from the ground-up designed around a specific, mature compiler stack be better able to integrate compilers in a more stable fashion than just shoe-horning static compilers into a very dynamic framework? ;)
Because JAX is not designed around a mature compiler stack. The history of Jax is more so that it matured alongside the compiler...
Pushback notwithstanding, this article is 100% correct in all PyTorch criticisms. PyTorch was a platform for fast experimentation with eager evaluation, now they shoehorn "compilers" into it. "compilers", because a lot of the work is done by g++ and Triton. It is a messy and quickly expanding codebase with many surprises like segfaults and leaks. Is scientific experimentation really sped up by these frameworks? Every…
... then they would get nothing done.
Pushback notwithstanding, this article is 100% correct in all PyTorch criticisms. PyTorch was a platform for fast experimentation with eager evaluation, now they shoehorn "compilers" into it. "compilers", because a lot of the work is done by g++ and Triton. It is a messy and quickly expanding codebase with many surprises like segfaults and leaks. Is scientific experimentation really sped up by these frameworks? Every…
> If researchers wrote directly in C or Fortran... ... then they would get nothing done.
Earlier quoted context omitted.
> If researchers wrote directly in C or Fortran... ... then they would get nothing done.
Why not?
From an eng/industry perspective, back in 2016/2017 I watched the realtime decline of Tensorflow towards Pytorch. The issue was TF had too many interfaces to accomplish the same thing and each one was rough in its own way. Along with some complexity for using serving and experiment logging via Tensorboard, but this wasn’t as bad at least for me. Keras was integrated in an attempt to help, but ultimately it wasn’t eno…
Imo the biggest issue (from memory) was that Tensorflow used a static computation graph. PyTorch was so much easier to work with.
Earlier quoted context omitted.
It's not about the syntax, it's all the knowledge, tools, existing code, etc that make Python so attractive.
I don't doubt that, but I'm specifically talking about new languages. I've seen far more enthusiasm from ML researchers for Mojo, which doesn't even do automatic differentiation, than for Dex. And to recycle an old HN comment of mine, people are much more eager to learn a functional programming language if it looks like NumPy (I'm talking about JAX here).
One aspect of jax that’s rarely touched on is browser stuff. Completely aside from deep learning, it’s straightforward to compile jax to a graphics shader you can call in js, which in this insane world is actually my preferred way to put numerical computing or linear algebra code on a web page.