Live data from Hacker News

PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

dev-discuss.pytorch.org

21–30 of 291 posts

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#21
I've been thinking a lot about trying to use functional dialect of Python like Coconut[0] and Hy[1] w/ JAX so I can write functional DL code.

Glad to see functorch[3] as PyTorch is the library I have the most experience with.

[0] http://coconut-lang.org/

[1] https://docs.hylang.org/en/alpha/

[2] https://github.com/google/jax

[3] https://github.com/pytorch/functorch

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#22

The more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy). It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language. I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.

What do you think the major drawbacks are? Speed would be the top of my list, but most projects to not need anything more than what python can currently pump out.

Basically everything. If I had to pick two things I hate the most it would be dynamic typing and Python's deployment story.

I wrote a somewhat tongue-in-cheek rant blog post. https://www.forrestthewoods.com/blog/things-i-like-about-pyt...

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#23
post #3

Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust. Rust has been eating C++ lunch. Same rapid rise of ecosystem story. Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in…

I don't know, x being a potential replacement for y doesn't say all that much. You could have been writing Java or C++ for the past 25 years and got loads of stuff done, solved problems, shipped software, made money etc.

Languages are fun to think about but you don't always need to be concerned with every vocal minority of programmers that like to talk about how their language is better than yours. Sometimes that replacement is better and sometimes those people are wrong. But even when they're right, being marginally better isn't that big of deal, or nearly enough to make for a viable rewrite or change of language.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#24

Neither language have proper tail call elimination, which, is absolutely insane to me. Yall really just write procedural code for everything?

if you want tail calls in Julia, there is a 3 line macro that gives it to you.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#25
post #15

> Julia says: > A language must compile to efficient code, and we will add restrictions to the language (type stability) to make sure this is possible. > A language must allow post facto extensibility (multiple dispatch), and we will organize the ecosystem around JIT compilation to make this possible. > The combination of these two features gives you a system that has dynamic language level flexibility (because you h…

Because people in that game value simplicity of use over anything else (performance, safety, maintenance) and Python is a top performer in that KPI while Java is not very good at this (though it is getting better but still)

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#26
post #14
post #3

Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust. Rust has been eating C++ lunch. Same rapid rise of ecosystem story. Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in…

Wow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too. I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.

> I'm not sure what advantage Julia

While I'm really not a fan of 1-based indexing, Julia's multiple dispatch is not something easy to match in Python.

[EDIT]: one thing that's still not solved in Julia is code startup time.

Many people will sell you some sort of workflow that works around the problem, but it's the same old tired arguments people would use to defend traditional compiled languages, and I'm not buying.

I really wish they would find a way to truly solve this.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#27

The more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy). It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language. I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.

What do you think the major drawbacks are? Speed would be the top of my list, but most projects to not need anything more than what python can currently pump out.

Aside from speed, one thing that really eats at me is that it makes any sort of functional programming overly verbose, unfun, and just not very idiomatic. Also the the vast majority of python programmers simply don't understand the best practices in their own ecosystem.

I was recently writing code using Reactor/RxJava in Java 11 w/ Lombok. I don't think I've ever been so productive or lead a team as productive as when we were going ham on the functional/reactive Java. Now that I'm back in Python land, I am constantly frustrate on a daily basis with both the language and the runtime at every turn. Even with the asyncio we are working on, it feels like the absolute minimum viable product when compared to the java, node, or rust I have done.

There are some fantastic python enhancements that bridge some of the gaps like PEP 484 (Type Hints) and PEP 517 (holy crap an actual build systems that are not dogcrap) but it feels like the python community does not care.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#28
post #14
post #3

Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust. Rust has been eating C++ lunch. Same rapid rise of ecosystem story. Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in…

Wow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too. I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.

Most definitely not single threaded. From one of my codes

# Threaded inner loop, each thread has no dependence upon others

  Threads.@threads for t=1:Nthr

        inner_gen_cpu1!(psum,ms,me,cls,2)

  end

That's all you need. You don't need pthread create/join, you don't need installable language extensions, you don't need to appeal to external tools/libraries to enable threading. Its built in to Julia. And it is trivial to use.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#29

The more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy). It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language. I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.

Bad languages like Python, JavaScript, PHP are responsible for powering large part of tech revolution. Ability to write bad code easily is IMO large part of why they’re so popular. Low barrier to entry helps to build huge ecosystem.

Sure, but I get the feeling that the pendulum is swinging from a period where being able to write code easily is relatively devalued to being able to read code easily. Ironically, python itself benefitted from the last swing of that pendulum, as it was widely regarded as an "easier to read" PL than, say C (well, yeah) or perl (well, super yeah) or PHP (super-duper yeah).

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#30
post #14
post #3

Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust. Rust has been eating C++ lunch. Same rapid rise of ecosystem story. Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in…

Wow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too. I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.

https://docs.julialang.org/en/v1/manual/multi-threading/
Post reply on HN