Earlier quoted context omitted.
The challenge comes from a person like me, who doesn't know off the top of my head which Greek letter ξ is. So for each of these symbols I'd have to google them and learn it, or have some notepad where I can copy paste the needed symbol
So you don't know the Greek alphabet, but write high-performance computing code involving non-trivial math? (Julia's main use case is HPC)
Jax vs. Julia (Vs PyTorch)
21–30 of 111 posts
Re: Jax vs. Julia (Vs PyTorch)
#22In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is c…
> but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I heartily disagree with that. Bikeshedding is about focussing on the trivial, and the symbols we choose in our codebases are hardly trivial, and indeed many of us regard naming things as one of the central problems in programming[1]. [1]: https://medium.com/hackernoon/naming-the-…
It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster to something that shines (in the words of the original author), while not having anything useful to say anything about the mathematical/technical aspects of the code itself.
Here is the definition of bikeshedding:
> The term was coined as a metaphor to illuminate Parkinson’s Law of Triviality. Parkinson observed that a committee whose job is to approve plans for a nuclear power plant may spend the majority of its time on relatively unimportant but easy-to-grasp issues, such as what materials to use for the staff bikeshed, while neglecting the design of the power plant itself, which is far more important but also far more difficult to criticize constructively. It was popularized in the Berkeley Software Distribution community by Poul-Henning Kamp[1] and has spread from there to the software industry at large.
Re: Jax vs. Julia (Vs PyTorch)
#23I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…
I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.
In Python, most libraries are big monoliths. Whereas in Julia, libraries are small and composable. Furthermore, it's the same language all the way down.
Python's libraries are superb, but the learning curve to develop (not to use them) is really steep.
Re: Jax vs. Julia (Vs PyTorch)
#24The most next gen autodiff library probably is https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety
Re: Jax vs. Julia (Vs PyTorch)
#25I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…
And if you ever do want to edit the code, you have to know the name of every non-ASCII symbol the codebase uses if you want to type out those same symbols without copying and pasting them. If you're not familiar with the material, entering a character like ξ can be a real challenge, and is actually more keystrokes than just typing "xi".
I’ll do it right here, by typing : ξ
Re: Jax vs. Julia (Vs PyTorch)
#26Earlier quoted context omitted.
Difference is 1 keystroke: you type \xi. Why is that a "real challenge"?
What editors does this work in?
Re: Jax vs. Julia (Vs PyTorch)
#27I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…
I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.
Re: Jax vs. Julia (Vs PyTorch)
#28Earlier quoted context omitted.
> but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I heartily disagree with that. Bikeshedding is about focussing on the trivial, and the symbols we choose in our codebases are hardly trivial, and indeed many of us regard naming things as one of the central problems in programming[1]. [1]: https://medium.com/hackernoon/naming-the-…
Unlike the example in the link you give, η isn't a generic random name like a,x that can mean anything. If you ever read a paper on stochastic gradient optimization, you'd know that η means learning rate in the context. It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster t…
Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context.
Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to mathematical notation puts up an unnecessary barrier to using the API/code and ultimately limits wider engagement/collaboration.
To attract a bigger slice of the ML community, choosing names that the ML hobbyyist can read, understand and use without pause is the better path forward.
Re: Jax vs. Julia (Vs PyTorch)
#29Earlier quoted context omitted.
> Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.
Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever…
Absolutely! Because as we all know, everyone speaks English.
The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.
Re: Jax vs. Julia (Vs PyTorch)
#30Earlier quoted context omitted.
I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.
superficial you say. How about I name these in chinese in my package?