Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The future of Deep Learning frameworks

neel04.github.io

Post reply on HN
Print view View original

81–90 of 113 posts

Re: The future of Deep Learning frameworks

#81

Post by ianbutler »

From an eng/industry perspective, back in 2016/2017 I watched the realtime decline of Tensorflow towards Pytorch.

The issue was TF had too many interfaces to accomplish the same thing and each one was rough in its own way. Along with some complexity for using serving and experiment logging via Tensorboard, but this wasn’t as bad at least for me.

Keras was integrated in an attempt to help, but ultimately it wasn’t enough and people started using Torch more and more even against the perception that TF was for prod workloads and Torch was for research.

TFA mentions the interface complexity as starting to be a problem with Torch, but I don’t think we’re anywhere near the critical point that would cause people to abandon it in favor of JAX.

Additionally with JAX you’re just shoving the portability problems mentioned down to XLA which brings its own issues and gotchas even if it hides the immediate reality of said problems from the end user.

I think the Torch maintainers should watch not to repeat the mistakes of TF, but I think theres a long way to go before JAX is a serious contender. It’s been years and JAX has stayed in relatively small usage.

Re: The future of Deep Learning frameworks

#82

Post by 6gvONxR4sf7o »

mccoyb wrote: post #31
funks_ wrote: post #24

I wish dex-lang [1] had gotten more traction. It’s JAX without the limitations that come from being a Python DSL. But ML researchers apparently don’t want to touch anything that doesn’t look exactly like Python. [1]: https://github.com/google-research/dex-lang

Dex is also missing user authored composable program transformations, which is one of JAX’s hidden superpowers. So not quite “JAX without limitations” — but certainly without some of the limitations.

This is both its strength and its weakness. As soon as you write a jaxpr interpreter, you lose all the tooling that makes the python interpreter so mature. For example stack traces and debugging become black holes. If jax made it easy to write these transformations without losing python’s benefits it would be incredible.

Re: The future of Deep Learning frameworks

#83

Post by markeroon »

ianbutler wrote: post #81

From an eng/industry perspective, back in 2016/2017 I watched the realtime decline of Tensorflow towards Pytorch. The issue was TF had too many interfaces to accomplish the same thing and each one was rough in its own way. Along with some complexity for using serving and experiment logging via Tensorboard, but this wasn’t as bad at least for me. Keras was integrated in an attempt to help, but ultimately it wasn’t eno…

Imo the biggest issue (from memory) was that Tensorflow used a static computation graph. PyTorch was so much easier to work with.

Re: The future of Deep Learning frameworks

#84

Post by ianbutler »

markeroon wrote: post #83
ianbutler wrote: post #81

From an eng/industry perspective, back in 2016/2017 I watched the realtime decline of Tensorflow towards Pytorch. The issue was TF had too many interfaces to accomplish the same thing and each one was rough in its own way. Along with some complexity for using serving and experiment logging via Tensorboard, but this wasn’t as bad at least for me. Keras was integrated in an attempt to help, but ultimately it wasn’t eno…

Imo the biggest issue (from memory) was that Tensorflow used a static computation graph. PyTorch was so much easier to work with.

Yup that was also definitely one among the many issues

Re: The future of Deep Learning frameworks

#86

Post by sundarurfriend »

Can we get the title changed to the actual title of the post? "The future of Deep Learning frameworks" sounds like a neutral and far wider-reaching article, and ends up being clickbait here (even if unintentionally).

"PyTorch is dead. Long live JAX." conveys exactly what the article about, and is a much better title.

Re: The future of Deep Learning frameworks

#87

Post by hedgehog »

funks_ wrote: post #49
hedgehog wrote: post #47

Earlier quoted context omitted.

It's not about the syntax, it's all the knowledge, tools, existing code, etc that make Python so attractive.

I don't doubt that, but I'm specifically talking about new languages. I've seen far more enthusiasm from ML researchers for Mojo, which doesn't even do automatic differentiation, than for Dex. And to recycle an old HN comment of mine, people are much more eager to learn a functional programming language if it looks like NumPy (I'm talking about JAX here).

Is Mojo actually getting significant uptake in research? I haven't been following closely but new tooling at that layer seems much more useful when cost-optimizing deployment.

Re: The future of Deep Learning frameworks

#88

Post by sundarurfriend »

dang wrote: post #56
munchler wrote: post #4

This is such a hyperbolic headline it's hard to be interested in reading the actual article.

We've unhyperbolized it via the subtitle.

I think that change was for the worse. (Just posted a comment asking for the original title: https://news.ycombinator.com/item?id=41275918)

The subtitle doesn't convey the content of the article nearly as well as the title does. Perhaps you can take a sentence like "PyTorch has been a net negative for scientific computing efforts" which the article does say, or some toned down versino of the original title, but the current title makes it sound like a very different article and felt like clickbait to me.

Re: The future of Deep Learning frameworks

#89

Post by 6gvONxR4sf7o »

One aspect of jax that’s rarely touched on is browser stuff. Completely aside from deep learning, it’s straightforward to compile jax to a graphics shader you can call in js, which in this insane world is actually my preferred way to put numerical computing or linear algebra code on a web page.

Re: The future of Deep Learning frameworks

#90

Post by spott »

FL33TW00D wrote: post #78

Are modern NN's really just static functions, and are they going to continue to be in the future? KV caching is directly in conflict with a purely functional approach.

Is it? The same inputs have the same outputs, the only thing different is the performance.

It is basically the same as memorization.

Post reply on HN