Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jax vs. Julia (Vs PyTorch)

kidger.site

Post reply on HN
Print view View original

31–40 of 111 posts

Re: Jax vs. Julia (Vs PyTorch)

#31

Post by ShamelessC »

tagrun wrote: post #12
animal_spirits wrote: post #11

Earlier quoted context omitted.

The challenge comes from a person like me, who doesn't know off the top of my head which Greek letter ξ is. So for each of these symbols I'd have to google them and learn it, or have some notepad where I can copy paste the needed symbol

So you don't know the Greek alphabet, but write high-performance computing code involving non-trivial math? (Julia's main use case is HPC)

What? Did you not read the parent of this thread?

> strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means

In any case, I program HPC stuff myself with pytorch and no - I don't know the Greek alphabet and probably don't understand "non trivial math". The assumption that these people can't contribute is pretty off-putting honestly. More engineers would join such efforts if there wasn't so much gatekeeping.

Re: Jax vs. Julia (Vs PyTorch)

#32

Post by longemen3000 »

I feel called out on the academic part hahaah. I simply want to code state of the art (thermodynamic) models, and at least julia helps by providing easy testing and publishing infraestructure. but obviously we can't compete with a corporation in code quality (we are trying!)

Unrelated, but for small sizes, i really prefer to use forward mode in julia (Via ForwardDiff.jl) instead of Zygote. the overhead of reverse ADing over an arbitrary function with mutation is not worth it.

Re: Jax vs. Julia (Vs PyTorch)

#33

Post by belval »

melissalobos wrote: post #29
belval wrote: post #20

Earlier quoted context omitted.

Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever…

> The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.

How many researchers in the ML/DL community don't speak English? I don't have hard numbers but I highly doubt that it's a significant proportion. What is the reach of your Japanese papers when almost no-one outside of Japan can read Japanese?

Even China, despite their best effort to de-westernize their culture still uses English in their research papers.

And if all the above wasn't enough, Julia's libraries are still all in English so if an hypothetical researcher's English is so poor that they don't know what "learning rate" is, I'd venture that they'll have trouble programming in Julia/JAX/PyTorch.

Re: Jax vs. Julia (Vs PyTorch)

#34

Post by tagrun »

00ajcr wrote: post #28
tagrun wrote: post #22

Earlier quoted context omitted.

Unlike the example in the link you give, η isn't a generic random name like a,x that can mean anything. If you ever read a paper on stochastic gradient optimization, you'd know that η means learning rate in the context. It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster t…

My interpretation of the point in the blog post was that explicitly spelling out variable names makes APIs and the underlying code much more accessible to a wider audience. Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context. Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to…

You are saying most people don't know what η in that context means (=people who likely haven't read a book or a paper on stochastic gradient, and don't know how it actually works), but they would somehow magically figure out what it actually does if we call it "learning_rate" in ASCII letters. How does that work?

FYI, the documentation of the function https://fluxml.ai/Flux.jl/stable/training/optimisers/ explicitly says it is learning rate:

> Learning rate (η): Amount by which gradients are discounted before updating the weights.

so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter.

Re: Jax vs. Julia (Vs PyTorch)

#35

Post by jstx1 »

nextos wrote: post #23
agumonkey wrote: post #17

Earlier quoted context omitted.

I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.

Exactly. I actually find Julia's ecosystem (not the language) way more approachable than Python's. In Python, most libraries are big monoliths. Whereas in Julia, libraries are small and composable. Furthermore, it's the same language all the way down. Python's libraries are superb, but the learning curve to develop (not to use them) is really steep.

I don't understand. What do you mean by "learning curve to develop" an existing Python library?

Re: Jax vs. Julia (Vs PyTorch)

#36

Post by 127 »

melissalobos wrote: post #29
belval wrote: post #20

Earlier quoted context omitted.

Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever…

> The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.

Google Translate is one click away. I can easily translate both Japanese and Chinese comments and variable names to get the gist of it. Using single hieroglyphs for it makes the entire endeavor impossible.

Re: Jax vs. Julia (Vs PyTorch)

#37

Post by SiempreViernes »

belval wrote: post #33
melissalobos wrote: post #29

Earlier quoted context omitted.

> The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.

How many researchers in the ML/DL community don't speak English? I don't have hard numbers but I highly doubt that it's a significant proportion. What is the reach of your Japanese papers when almost no-one outside of Japan can read Japanese? Even China, despite their best effort to de-westernize their culture still uses English in their research papers. And if all the above wasn't enough, Julia's libraries are still…

How many don't speak it as a native language? Quite a lot as most of the world uses something else as their primary language.

If you're instead asking of how many can struggle trough an english text supported by machine translators, then that's clearly almost everyone.

There's very often a significant gap between the ease with which the native and the foreign language can be used for reasoning, but surely I don't need to point that out since any bilingual person knows this.

Re: Jax vs. Julia (Vs PyTorch)

#38

Post by tagrun »

longemen3000 wrote: post #32

I feel called out on the academic part hahaah. I simply want to code state of the art (thermodynamic) models, and at least julia helps by providing easy testing and publishing infraestructure. but obviously we can't compete with a corporation in code quality (we are trying!) Unrelated, but for small sizes, i really prefer to use forward mode in julia (Via ForwardDiff.jl) instead of Zygote. the overhead of reverse ADi…

In the context of neural networks with differential equations (which appears to be the original poster's field), the trade-off depends: https://diffeqflux.sciml.ai/dev/ControllingAdjoints/

Re: Jax vs. Julia (Vs PyTorch)

#39

Post by jstx1 »

tagrun wrote: post #34
00ajcr wrote: post #28

Earlier quoted context omitted.

My interpretation of the point in the blog post was that explicitly spelling out variable names makes APIs and the underlying code much more accessible to a wider audience. Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context. Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to…

You are saying most people don't know what η in that context means (=people who likely haven't read a book or a paper on stochastic gradient, and don't know how it actually works), but they would somehow magically figure out what it actually does if we call it "learning_rate" in ASCII letters. How does that work? FYI, the documentation of the function https://fluxml.ai/Flux.jl/stable/training/optimisers/ explicitly s…

> How does that work?

You can look up "learning rate" much easier than to look up "what is this Greek letter on my screen" followed by "what is the use of this Greek letter in my context" and only then followed by searching for "learning rate"

More importantly, it's possible to know what a learning rate is without knowing what Greek letter it's commonly denoted as. Especially since mathematical notation is so inconsistent across authors. I want less ambiguity in code, not more. Explicit is better than implicit.

Mathematical notation is notorious for being an absolute mess of inconsistencies. Who in their right mind looked at it and went "yep, I want more of this in my source code".

Re: Jax vs. Julia (Vs PyTorch)

#40

Post by blindseer »

Another massive frustration for me is that Julia has no formal way to say "here are public functions and these are private" but does have a completely orthogonal way of saying "these are functions that will populate your global namespace if you use `using`", i.e. `export variable_name`, and people absolutely confuse the hell out of these. I don't think there's even agreement in the Julia community if you should use `export` for your public API or not.

And if you misspell the exports or change the variable in question, Julia won't even warn you about it. That is straight crazy behavior to me, and I still don't understand how that hasn't been changed.

The `using` + `import` packages in Julia combined with how `export`s work make it SUCH a confusing and frustrating experience for beginners in Julia.

I personally like mathematical symbols when I'm writing and reading code in my domain, but I do feel very lost when I'm reading Julia code outside of my area of expertise. All my colleagues hate it too (hard to grep, hard to type if you don't know the math and are just a software engineer) and I'm coming around to the idea of not using it or documenting it explicitly.

The fact that mutable structs are easier to use but immutable structs are more performant, the lack of composition of fields the way Go handles it, the lack of traits or interfaces, the sorry state of compilation time, the non existent tooling all lead a beginner / intermediate Julia developer in the wrong direction in my opinion. It's very easy to write code that is straight up broken or just not efficient in Julia, and that's probably why I won't pick it for a big project going forward.

But I'm still keeping my eye on it. Maybe in 5 years it'll be the language for a lot of the jobs?

Post reply on HN