Live data from Hacker News

The future of Deep Learning frameworks

neel04.github.io

31–40 of 113 posts

Re: The future of Deep Learning frameworks

#31
post #24

I wish dex-lang [1] had gotten more traction. It’s JAX without the limitations that come from being a Python DSL. But ML researchers apparently don’t want to touch anything that doesn’t look exactly like Python. [1]: https://github.com/google-research/dex-lang

Dex is also missing user authored composable program transformations, which is one of JAX’s hidden superpowers.

So not quite “JAX without limitations” — but certainly without some of the limitations.

Re: The future of Deep Learning frameworks

#32
post #14

Earlier quoted context omitted.

You don't think its possible to (even theoretically) improve on javascript for its intended use case? What a terrific lack of imagination. Typescript and Elm would like a word

No, I said there's nothing that exists right now that's theoretically better. Typescript isn't. It'd be great if TS's type inference were smart enough that it basically takes no additional dev input vs JS, but until then, it's expensive to use. It's also bolted on awkwardly, but that's changing soon. Could also imagine JS getting some nice Py features like list comp. Also, generally when people complain that JS won t…

I see this a lot, and I want to find the right words for it: I don't want the types automatically determined for what I write, because I write mistakes.

I want to write the type, and for that to reveal the mistake.

Re: The future of Deep Learning frameworks

#33
> I’ve personally known researchers who set the seeds in the wrong file at the wrong place and they weren’t even used by torch at all - instead, were just silently ignored, thus invalidating all their experiments. (That researcher was me)

Some assert-ing won't hurt you. Seriously. It might even help keeping your sanity.

Re: The future of Deep Learning frameworks

#34
post #16
post #9

Jax is dead, long live PyTorch. PyTorch has _twenty times_ as many users as Jax. Any rumors of its death are highly exaggerated

They used to say the same thing about Perl and Python Downvoted. Hmmm. I’m a little tired so I don’t want to go into detail. However, I was a Perl programmer when Python was rising. So, needless to say, having a big lead doesn’t matter. Please learn from history. A big lead means nothing.

It’s been years and Jax is just where it was, no growth whatsoever. And that’s with all of Google forced internally to use only Jax. Look, I like the technical side of Jax for the most part, but it’s years too late to the party and it’s harder to use than PyTorch. It just isn’t going to ever take off at this point.

Re: The future of Deep Learning frameworks

#35

Earlier quoted context omitted.

No, I said there's nothing that exists right now that's theoretically better. Typescript isn't. It'd be great if TS's type inference were smart enough that it basically takes no additional dev input vs JS, but until then, it's expensive to use. It's also bolted on awkwardly, but that's changing soon. Could also imagine JS getting some nice Py features like list comp. Also, generally when people complain that JS won t…

I see this a lot, and I want to find the right words for it: I don't want the types automatically determined for what I write, because I write mistakes. I want to write the type, and for that to reveal the mistake.

If you explicitly mark types at lower levels, or just use typed libs, it's not very easy to pass the wrong type somewhere and have it still accidentally work. The most automatic type inference I can think of today is in Rust, which is all about safety, or maybe C++ templates count too.

Re: The future of Deep Learning frameworks

#36
post #9

Jax is dead, long live PyTorch. PyTorch has _twenty times_ as many users as Jax. Any rumors of its death are highly exaggerated

It's definitely exaggerated, but I personally prefer JAX and have found it easier to use than PyTorch for almost everything. If you haven't already, I would give JAX a good try.

Re: The future of Deep Learning frameworks

#37
post #28

i like pytorch because all the academia release their code with it ive never even heard of jax nor will i have the skills to use it i literally just want to know two things: 1) how much vram 2) how to run it on pytorch

Jax is a competing computational framework that does something similar to PyTorch, so both of your questions don't really make sense.

Maybe deisteve will answer for himself, but I don't think that's meant to mean how to run Jax on Pytorch, but rather the questions he's interested in for any published model.

Re: The future of Deep Learning frameworks

#38
My main reason to avoid Jax is Google. Google doesn't provide good support even for things you pay them for. They do things because they want to, to get their internal promotions, irrespective of their customers or the impact on them.

Re: The future of Deep Learning frameworks

#39
post #24

I wish dex-lang [1] had gotten more traction. It’s JAX without the limitations that come from being a Python DSL. But ML researchers apparently don’t want to touch anything that doesn’t look exactly like Python. [1]: https://github.com/google-research/dex-lang

It seems like an experimental research language. Julia also competes in this domain from a more practical standpoint and has less limitations than JAX as I understand it, but is less mature and still working on getting wider traction.

The Julia AD ecosystem is very interesting in that the community is trying to make the entire language differentiable, which is much broader in scope than what Torch and JAX are doing. But unlike Dex, Julia is not a language built from the ground up for automatic differentiation.

Shameless plug for one of my talks at JuliaCon 2024: https://www.youtube.com/live/ZKt0tiG5ajw?t=19747s. The comparison between Python and Julia starts at 5:31:44.

Re: The future of Deep Learning frameworks

#40
post #31
post #24

I wish dex-lang [1] had gotten more traction. It’s JAX without the limitations that come from being a Python DSL. But ML researchers apparently don’t want to touch anything that doesn’t look exactly like Python. [1]: https://github.com/google-research/dex-lang

Dex is also missing user authored composable program transformations, which is one of JAX’s hidden superpowers. So not quite “JAX without limitations” — but certainly without some of the limitations.

Are you talking about custom VJPs/JVPs?
Post reply on HN