Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jax vs. Julia (Vs PyTorch)

kidger.site

Post reply on HN
Print view View original

41–50 of 111 posts

Re: Jax vs. Julia (Vs PyTorch)

#41

Post by dman »

blindseer wrote: post #14
tagrun wrote: post #7

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is c…

I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least…

Just trying to use this thread for a market demand survey - Julia devs, would you pay $9.99 a month for better tooling? [Maybe responses to this will encourage devs to notice that there is a viable market here?]

Re: Jax vs. Julia (Vs PyTorch)

#42

Post by tagrun »

ShamelessC wrote: post #31
tagrun wrote: post #12

Earlier quoted context omitted.

So you don't know the Greek alphabet, but write high-performance computing code involving non-trivial math? (Julia's main use case is HPC)

What? Did you not read the parent of this thread? > strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means In any case, I program HPC stuff myself with pytorch and no - I don't know the Greek alphabet and probably don't understand "non trivial math". The assumption that these people can't contribute i…

So you think there's a cabal of "gatekeeper" scientists who are trying to hold the hordes of capable (yet somehow unfamiliar with the corresponding mathematical literature) engineers from making useful contributions to their libraries, because they use the conventional notation from the scientific literature?

They should instead tread carefully and dispose of all Greek letters in their code to ensure you are not put off, because that's the real reason that stops you from making useful contributions?

Re: Jax vs. Julia (Vs PyTorch)

#43

Post by machinekob »

aaplok wrote: post #16
machinekob wrote: post #5

I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…

> Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.

I can assure you that more people exist with B1 knowledge of english then folks that have at least minimum knowledge of all math and computer science related domains that are used for deep learning (even subset like audio/video) even if we use only using software engineers/ml engineers as reference you have so much domain knowledge needed to even consider reading notation (without proper explanation in docs).

And most of the variables in code still will be in english or other natural language so it is just easier to read english then read math notation mixed with english/other language for MOST (not all) people using underlying software.

Re: Jax vs. Julia (Vs PyTorch)

#44

Post by omegalulw »

melissalobos wrote: post #29
belval wrote: post #20

Earlier quoted context omitted.

Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever…

> The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.

> Absolutely! Because as we all know, everyone speaks English.

I understand the sentiment here, especially as English is not my native language, but for many domains, for anything close to the state of the art, English is lingua franca.

Re: Jax vs. Julia (Vs PyTorch)

#45

Post by jstx1 »

aaplok wrote: post #16
machinekob wrote: post #5

I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…

> Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.

> Naming things by their English name is not more universal than using Greek letters.

It is - the people who understand the Greek-letter notation will understand the English words but the inverse statement isn't always true.

Re: Jax vs. Julia (Vs PyTorch)

#46

Post by tagrun »

jstx1 wrote: post #39
tagrun wrote: post #34

Earlier quoted context omitted.

You are saying most people don't know what η in that context means (=people who likely haven't read a book or a paper on stochastic gradient, and don't know how it actually works), but they would somehow magically figure out what it actually does if we call it "learning_rate" in ASCII letters. How does that work? FYI, the documentation of the function https://fluxml.ai/Flux.jl/stable/training/optimisers/ explicitly s…

> How does that work? You can look up "learning rate" much easier than to look up "what is this Greek letter on my screen" followed by "what is the use of this Greek letter in my context" and only then followed by searching for "learning rate" More importantly, it's possible to know what a learning rate is without knowing what Greek letter it's commonly denoted as. Especially since mathematical notation is so inconsi…

You are not answering the (rhetorical) question that you quoted, and the answer to your response is already in the paragraph that followed it:

As I said, the necessary keywords for Googling it, along with a brief description is already present in the documentation.

The quibble here is about the necessity of reproducing the all the necessary keywords for an accurate Google search during every single function call.

> Mathematical notation is notorious for being an absolute mess of inconsistencies.

According to whom? What exactly is inconsistent?

Re: Jax vs. Julia (Vs PyTorch)

#47

Post by belval »

SiempreViernes wrote: post #37
belval wrote: post #33

Earlier quoted context omitted.

How many researchers in the ML/DL community don't speak English? I don't have hard numbers but I highly doubt that it's a significant proportion. What is the reach of your Japanese papers when almost no-one outside of Japan can read Japanese? Even China, despite their best effort to de-westernize their culture still uses English in their research papers. And if all the above wasn't enough, Julia's libraries are still…

How many don't speak it as a native language? Quite a lot as most of the world uses something else as their primary language. If you're instead asking of how many can struggle trough an english text supported by machine translators, then that's clearly almost everyone. There's very often a significant gap between the ease with which the native and the foreign language can be used for reasoning, but surely I don't nee…

The programming language is already in English, implying that using greek letters to map to math concepts is easier to understand for non-native is disingenuous.

Re: Jax vs. Julia (Vs PyTorch)

#48

Post by manwe150 »

dman wrote: post #41
blindseer wrote: post #14

Earlier quoted context omitted.

I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least…

Just trying to use this thread for a market demand survey - Julia devs, would you pay $9.99 a month for better tooling? [Maybe responses to this will encourage devs to notice that there is a viable market here?]

That question is rather cryptic. Your profile LinkedIn url also might be broken? (For added mystery maybe)

Re: Jax vs. Julia (Vs PyTorch)

#49

Post by mhh__ »

martinsmit wrote: post #24
SemanticStrengh wrote: post #6

The most next gen autodiff library probably is https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety

Idk, Enzyme is pretty next gen, all the way down to LLVM code. https://github.com/EnzymeAD/Enzyme

So enzyme is pretty cool but IMO the documentation is terrible and the API is absolutely insane.

Re: Jax vs. Julia (Vs PyTorch)

#50

Post by longemen3000 »

tagrun wrote: post #38
longemen3000 wrote: post #32

I feel called out on the academic part hahaah. I simply want to code state of the art (thermodynamic) models, and at least julia helps by providing easy testing and publishing infraestructure. but obviously we can't compete with a corporation in code quality (we are trying!) Unrelated, but for small sizes, i really prefer to use forward mode in julia (Via ForwardDiff.jl) instead of Zygote. the overhead of reverse ADi…

In the context of neural networks with differential equations (which appears to be the original poster's field), the trade-off depends: https://diffeqflux.sciml.ai/dev/ControllingAdjoints/

yeah, my systems are really small in comparison (1-20) but with higher order derivatives (up to 4th order), so reverse AD is not the best in that regard
Post reply on HN