Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jax vs. Julia (Vs PyTorch)

kidger.site

Post reply on HN
Print view View original

61–70 of 111 posts

Re: Jax vs. Julia (Vs PyTorch)

#61

Post by wmoses »

mhh__ wrote: post #54
wmoses wrote: post #51

Earlier quoted context omitted.

Enzyme dev here. The documentation is not just bad, it is in fact actively missing for several things (like forward mode AD, split mode, recent vector). Having recently finished up a number of big technical challenges (forward, vector, split, BLAS), we're hoping to have a big documentation-athon at the end of summer (all are welcome)!

Interesting. I'm mulling over integrating Enzyme into the LLVM D compiler so I may/will join in with that.

Do it! Would be happy to help. Also we have weekly development meetings (time about to change for summer), which you’re welcome to join.

Also feel free to email me (wmoses@mit.edu), would be happy to chat and or help bootstrap.

Re: Jax vs. Julia (Vs PyTorch)

#62

Post by mcabbott »

nullstyle wrote: post #57
tagrun wrote: post #46

Earlier quoted context omitted.

You are not answering the (rhetorical) question that you quoted, and the answer to your response is already in the paragraph that followed it: As I said, the necessary keywords for Googling it, along with a brief description is already present in the documentation. The quibble here is about the necessity of reproducing the all the necessary keywords for an accurate Google search during every single function call. > M…

> The quibble here is about the necessity of reproducing the all the necessary keywords for an accurate Google search during every single function call. No, that is not the quibble. My quibble is with choosing identifiers that make code less legible when taken by itself. The best code teaches future readers about how it works.

The struct's field name is `eta`, but this is an internal detail. Its constructor takes a positional-only argument, no public name.

The greek letter is used in the documentation. And the reason is that every optimiser's documentation links to the original paper, and tries to follow that. If the Adam paper calls the two decay rates β1, β2, then staying close to that seems the least confusing option.

Re: Jax vs. Julia (Vs PyTorch)

#63

Post by patrick451 »

tagrun wrote: post #7

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is c…

> In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not.

> Item #2, again another Zygote bug.

If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the depencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme.

https://github.com/FluxML/Flux.jl/blob/master/Project.toml

https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.t...

Re: Jax vs. Julia (Vs PyTorch)

#65

Post by make3 »

alphachloride wrote: post #64

One of the biggest hurdles to my use of Jax is that it doesn't work on Windows, my daily driver. It's inconvenient when playing around with a new tool that requires a whole other OS to run. I'm hoping that gets figured out soon.

does it not work on WSL ?

Re: Jax vs. Julia (Vs PyTorch)

#66

Post by SemanticStrengh »

wmoses wrote: post #59
SemanticStrengh wrote: post #53

Earlier quoted context omitted.

On HN I am often seeing libraries/blog authors react very quickly to references of their projects. I'm curious, how do you achieve this, is it a coincidence and where you reading HN comments, or do you have a crwaler/API than send you a notification when your project github link is found in a new HN comment ?

I was actually socializing with folks at the Cambridge Area Julia meetup where I gave a talk on Enzyme’s Julia bindings no less when someone shared this. So in this case pure coincidence.

Thanks for the data point :)

Re: Jax vs. Julia (Vs PyTorch)

#67

Post by Tarrosion »

blindseer wrote: post #14
tagrun wrote: post #7

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is c…

I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least…

There's something a little strange and subjective about saying "a problem with language X is that third party package Y has a bug." Y != X. If Y is some tiny package that hardly anyone uses or cares about, then bugs in Y don't imply much about the typical experience of using X. But if Y is a very common package, practically essential to everyday workflows, then bugs in Y do have implications for the typical X user.

The subjectivity arises in deciding whether some third party package Y is totally common, essential, de facto part of the language itself, or esoteric, negligible, who cares. I perceive that a lot of the back and forth around "is Julia good" basically boils down to this.

"I don't like Julia! I tried to use it and had an annoying bug!" "Okay, but isn't part of Julia itself or even written/maintained by core Julia contributors, so bugs therein are irrelevant for the awesome multiple dispatch speedy beauty of Julia!" "Hard disagree, is the de facto standard for and maintained by The parties are just talking past each other here; there's no ground truth. We'd all be better off if we were more explicit about whether comments applied to (a) the core language itself (b) core language and standard lib (c) the universal experience of working in Julia, i.e. packages that nearly everyone encounters (or lack thereof) (d) packages which are key for certain kind of work but irrelevant for others; auto differentiation would seem to go here (e) niche packages.

Disclaimers?: I love Julia-the-language; it is my favorite programming language by a mile. My brain seems to work similarly to Julia-the-language, so programming in Julia feels fluid and effortless to me; no other language sparks joy the same way. But Julia-the-ecosystem certainly has its holes and weak spots. And the limited size of Julia-the-community means even relatively prominent packages can be less maintained than e.g. Python analogues. Example 1: it's unnerving the extent to which the Julia data ecosystem rests on the heroic efforts of a few people [1]. Example 2: even relatively mainstream packages can have issues and PRs unaddressed indefinitely, e.g. I like to plot with Gadfly.jl but am nonetheless frustrated that I've had an open issue there and an upstream PR for over a year.

[1] https://github.com/orgs/JuliaData/people

Re: Jax vs. Julia (Vs PyTorch)

#68

Post by mountainriver »

blindseer wrote: post #40

Another massive frustration for me is that Julia has no formal way to say "here are public functions and these are private" but does have a completely orthogonal way of saying "these are functions that will populate your global namespace if you use `using`", i.e. `export variable_name`, and people absolutely confuse the hell out of these. I don't think there's even agreement in the Julia community if you should use `…

Yeah I really want to like Julia but for all the reasons you mentioned I’ll just use python. Also python is making a ton of improvements and looks better by the day

Re: Jax vs. Julia (Vs PyTorch)

#69

Post by mountainriver »

blindseer wrote: post #14
tagrun wrote: post #7

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is c…

I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least…

Yup Julia looks nice on paper but it’s devx is really bad

Re: Jax vs. Julia (Vs PyTorch)

#70

Post by mattkrause »

jstx1 wrote: post #39
tagrun wrote: post #34

Earlier quoted context omitted.

You are saying most people don't know what η in that context means (=people who likely haven't read a book or a paper on stochastic gradient, and don't know how it actually works), but they would somehow magically figure out what it actually does if we call it "learning_rate" in ASCII letters. How does that work? FYI, the documentation of the function https://fluxml.ai/Flux.jl/stable/training/optimisers/ explicitly s…

> How does that work? You can look up "learning rate" much easier than to look up "what is this Greek letter on my screen" followed by "what is the use of this Greek letter in my context" and only then followed by searching for "learning rate" More importantly, it's possible to know what a learning rate is without knowing what Greek letter it's commonly denoted as. Especially since mathematical notation is so inconsi…

This depends a lot on the target audience for your code.

For research-focused code, it is likely that whatever you're implementing was initially described in terms of mathematical notation (e.g., in a paper or book). It can be helpful to have variables that unambiguously match that canonical source. In fact, a lot of my Julia code has docstrings containing references/links to the original paper and a comment noting that it uses the notation therein.

This sidesteps the problem where textual descriptions like `learning_rate` can sometimes be ambiguous: is it the original learning rate, or perhaps the current rate after applying some sort of schedule or decay? I think the Flux documentation is pretty close to ideal, in that it's got a symbol you can match against equations (though no reference to them) as well as text that you can search to learn more.

Post reply on HN