Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jax vs. Julia (Vs PyTorch)

kidger.site

Post reply on HN
Print view View original

91–100 of 111 posts

Re: Jax vs. Julia (Vs PyTorch)

#91

Post by dklend122 »

There are several repeating (valid) critiques. There is also a sense that things won't get better. I don't think that's necessarily the case.

I can't give timelines, but I'll list the themes and point to the very active work going on in those areas:

1. Static analysis/ it's hard to write large codebases of correct Julia code.

That's being addressed on several fronts:

a. Work on both traits/interfaces and sound static typing is progressing in Jan Vitek's group at Northwestern

b. User extensible type lattice for correctness proof of programs. See here: https://news.ycombinator.com/item?id=26136212

c. Jet.jl already catches method errors at JIT time. It's under-used IMO.

2. Compilation time and package dev latency.

a. We're getting close to native code caching, and more: https://discourse.julialang.org/t/precompile-why/78770/8

As you'll also read, the difficulty is due to important tradeoffs Julia made with composability and aggressive specialization...but it's not fundamental and can be surmounted. Yes there's been some pain, but in the end hopefully we'll have something approximating the best of both worlds.

b. Runtime free static compilation: https://www.youtube.com/watch?v=YsNC4oO0rLA

c. Semantics for separate compilation: https://gist.github.com/JeffBezanson/dd86043ef867954bd7e2163...

Some combination of the above should also address deployment scenarios, whether that be CLI, mobile, browser or even embedded (Yes, with an (expanding) subset of code, StaticCompiler can produce binaries down to the 10s of Kbs with real working Julia programs that use non trivial abstractions like subtyping and higher order functions)

3. Zygote is rough

a. Enzyme will do (almost) whole language AD, including mutation.

b. Diffractor.jl

Re: Jax vs. Julia (Vs PyTorch)

#92

Post by dklend122 »

blindseer wrote: post #72
Tarrosion wrote: post #67

Earlier quoted context omitted.

There's something a little strange and subjective about saying "a problem with language X is that third party package Y has a bug." Y != X. If Y is some tiny package that hardly anyone uses or cares about, then bugs in Y don't imply much about the typical experience of using X. But if Y is a very common package, practically essential to everyday workflows, then bugs in Y do have implications for the typical X user. T…

I partly agree with you, in that the author is mainly complaining about Flux and other packages they've used. But on the other hand, all the reasons the author has listed as annoyances I've experienced the exact same things and more. And I'm definitely not blaming the package maintainers for this. There's either 1) something wrong with the Julia language that results in code that is harder to maintain at scale, or 2)…

Please see my comment here: https://news.ycombinator.com/item?id=31269739

Based on the above, I expect more than marginal improvements in these areas. Would you agree?

Re: Jax vs. Julia (Vs PyTorch)

#93

Post by nullstyle »

mcabbott wrote: post #62
nullstyle wrote: post #57

Earlier quoted context omitted.

> The quibble here is about the necessity of reproducing the all the necessary keywords for an accurate Google search during every single function call. No, that is not the quibble. My quibble is with choosing identifiers that make code less legible when taken by itself. The best code teaches future readers about how it works.

The struct's field name is `eta`, but this is an internal detail. Its constructor takes a positional-only argument, no public name. The greek letter is used in the documentation. And the reason is that every optimiser's documentation links to the original paper, and tries to follow that. If the Adam paper calls the two decay rates β1, β2, then staying close to that seems the least confusing option.

Perhaps I'm missing your point, but I think you're focussing too much on the specific case that someone who isn't me came up with.

My most general point is that the identifiers we use in our code are almost never just convention or taste when we are sharing that code with anyone else (and for most, "anyone else" includes our future selves). Getting a little more specific, I'm specifically interested in Julia and look forward to working in it further, but I've personally felt pain around scientific/mathematical notation when trying to understand code I've found on github. tagrun dismissing my pain as nonsense and the people who argue for my ilk as perpetrators of bikesheddding is dipshitted. Yeah, I'm probably the asshole for being a college dropout trying to leverage modern scientific computing for my own ends (snoogins), but I'm also willing to bet tagrun is probably the member of a team that talks down to junior members and complains they haven't read enough papers or the right papers to see the magnificience of their code ;).

It's fine to write code that demands a domain expert to understand, but don't pretend like its good across all dimensions. There are tradeoffs involved.

Personally I find the preponderance of scientific/mathematical notation (whatever you want to call it) in Julia to be cute; It certainly does bind the code to linked papers in a pretty cool way when it all fits together properly. That said, its a pain in the ass when it doesn't fit together properly and I've personally had a journey into Julia spoiled due to frequency at which I had to figure out how to notate something or what word to use when regarding some squiggle I haven't encountered before. I look forward to having a better intuition for the greek alphabet but until then Julia will often be harder to read, let alone understand when compared to ruby or javascript or go or C# or any other of the roughly dozen programming languages I've worked with and feel comfortable translating between.

Re: Jax vs. Julia (Vs PyTorch)

#94

Post by agumonkey »

abakus wrote: post #27
agumonkey wrote: post #17

Earlier quoted context omitted.

I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.

superficial you say. How about I name these in chinese in my package?

You're annoyed to the point of overreaction I'd say. I assume julia's devs use symbols that are culturally set already. Not just random glyphs for no good reason.

I've seen this in math, I come the dev land around java peak, where it was a holy trait to write runComputationThatIsMaybeSafeOverArguments(...), because that's where they though information was. In math they lived in the opposite side of things.. names were few and concision was more important. Also structural thinking made names really unimportant, mostly redundancy and waste of time, the domain and properties were.

I may be wrong though.

Re: Jax vs. Julia (Vs PyTorch)

#95

Post by 411111111111111 »

snicker7 wrote: post #90
socialdemocrat wrote: post #78

Earlier quoted context omitted.

I use export for my public functions. What is wrong with that? How exactly does Python signal what is public? Nothing but conventions like underscore, which btw Julia developers use as well. If I really really don't want somebody to use a function, I might underscore it but generally I don't see the point.

The underscore naming convention is far from ubiquitous in either the Python or Julia communities. The more practical indicator of a function being "public" is whether or not it appears in the published API documentation. Unlike Julia, Python projects are more likely to have complete documentation (or even documentation at all). So there is a better understanding of what "public" means in Python. Without docs, it's a…

There are effectively private functions in python though.

Double underscores before the name makes it impossible to call them from the outside. It's more like typescripts private however, so it's still technically possible to call these functions in a specific way, it just won't happen on accident.

Re: Jax vs. Julia (Vs PyTorch)

#96

Post by g42gregory »

>> Where does JAX shine vs. Julia? ...Documentation...Code Quality...

I am not sure I need to go beyond that. The decision is somewhat easy here. I am surprised that integration is not mentioned. JAX is just Python (library) so it could be easily integrated into the rest of the system. It fits very well into Python/C skillset.

Re: Jax vs. Julia (Vs PyTorch)

#97

Post by ur-whale »

aaplok wrote: post #16
machinekob wrote: post #5

I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/function…

> Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.

You're entirely ignoring the size of the bubbles.

Re: Jax vs. Julia (Vs PyTorch)

#98

Post by ur-whale »

tagrun wrote: post #9
bobbylarrybobby wrote: post #8

Earlier quoted context omitted.

And if you ever do want to edit the code, you have to know the name of every non-ASCII symbol the codebase uses if you want to type out those same symbols without copying and pasting them. If you're not familiar with the material, entering a character like ξ can be a real challenge, and is actually more keystrokes than just typing "xi".

Difference is 1 keystroke: you type \xi. Why is that a "real challenge"?

Sure, and that works great with rg and grep.

Re: Jax vs. Julia (Vs PyTorch)

#99

Post by dunefox »

127 wrote: post #36
melissalobos wrote: post #29

Earlier quoted context omitted.

> The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.

Google Translate is one click away. I can easily translate both Japanese and Chinese comments and variable names to get the gist of it. Using single hieroglyphs for it makes the entire endeavor impossible.

So, you are fine with google translating variable names but you absolutely cannot interpret symbols? Wow.

Re: Jax vs. Julia (Vs PyTorch)

#100

Post by doliveira »

belval wrote: post #47
SiempreViernes wrote: post #37

Earlier quoted context omitted.

How many don't speak it as a native language? Quite a lot as most of the world uses something else as their primary language. If you're instead asking of how many can struggle trough an english text supported by machine translators, then that's clearly almost everyone. There's very often a significant gap between the ease with which the native and the foreign language can be used for reasoning, but surely I don't nee…

The programming language is already in English, implying that using greek letters to map to math concepts is easier to understand for non-native is disingenuous.

It's really not. There are just a few programming language-specific keywords that you have to memorize, unlike infinite possible combinations of English words for parameters.

edit: as an example, when programming for Brazilian businesses, we generally just use a mix and match of Portuguese and English like getFinanciamento. "Get" is a technical concept while "Financiamento" is a business concept and it often doesn't make sense to translate either. Best just to leave them both in their respective domain-native language.

In case of maths, Greek IS the domain native language. Now, I wouldn't necessarily enjoy a codebase with Unicode variables for other reasons, but using Greek is best to map directly from the equations.

Post reply on HN