Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

dev-discuss.pytorch.org

Post reply on HN
Print view View original

241–250 of 291 posts

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#241

Post by cookieater »

dunefox wrote: post #187
rg111 wrote: post #143

Earlier quoted context omitted.

With the slightest hint arising that Julia would be the future of ML and DL, I learned it. But, then what? I could not use it anywhere I worked. The ecosystem was lacking. Julia is good, but for what exactly? People involved with Julia are always big with words, but when will I see it in use somewhere?

The ecosystem is a superset of Pythons: https://github.com/JuliaPy/PyCall.jl

It's probably more fair to claim that R/Python/Julia have the same ecosystem now if you are willing to deal with speed bumps and some clunky interfaces. Because there are packages going in both direction for all three.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#242

Post by ddragon »

bjourne wrote: post #236
elcritch wrote: post #174

Earlier quoted context omitted.

It's real kernel threads. Julia also supports cluster computing too.

According to this discussion thread, Julia requires the number of kernel threads to be specified at startup: https://discourse.julialang.org/t/does-multithreading-requir... This seem to be more like pyprocessing and much more limited than the pthreads interface.

I didn't go deep on Julia's multithreading, but what he is saying is that Julia uses an MxN threading (I think nowadays if you don't specify it at startup it will just use one for each cpu thread), which is the same as a language I did most of distributed programming (elixir/erlang), and as far as I know it's the same as Go.

Having 1 kernel thread for each CPU thread means that your program can use all available CPU threads at the same time (so you get all the parallelism available within the machine), and having a language based scheduler for each thread means you can have minimal overhead (no need to do a system call) to create a new concurrent execution (meaning lightweight/green threading similar to what python allows, except being automatically distributed by the language within all kernel/cpu threads). In Elixir this means you can create millions of processes even though the OS will only see one thread per logical cpu thread, and I never felt the limitation of this abstraction over multiprocessing (of course, Julia is definitely nowhere near as mature - and maybe never will due to stuff like preemptive scheduling and parallel garbage collection that is easier to implement in a language with only immutable types, though it seems to be moving along, and in Julia 1.7 the processes being able to move between kernel threads solving the issue mentioned in that discussion you linked).

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#243

Post by ChrisRackauckas »

jonniedie wrote: post #240
ChrisRackauckas wrote: post #200

Earlier quoted context omitted.

> Zero based indexing makes zero sense The sensibility of the index choice is equal to the starting value of the index.

StarWars indexing makes 4 sense.

yupyupyup, you got it. :)

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#244

Post by hpcjoe »

laichzeit0 wrote: post #148
hpcjoe wrote: post #16

Earlier quoted context omitted.

This is one of the nicer aspects of Julia. It starts out being a great language to work in. Its easy to implement algorithms that are generally difficult in other languages. Its important to remember that most of the python ecosystem, isn't written in python. The functions are often thin wrappers/objects around the real computation, which is often written in a faster language, C/C++/Fortran. Julia excels in composabi…

Let’s not ignore the giant elephant in the room: 1-based indexing. I don’t particularly care since I use R and Python but Java, C, C/C++, C# all used 0-based indexing. It’s truly a bizarre choice Julia made there.

1-based indexing makes sense to computational scientists (target audience). Fortran, Matlab, etc. make very good use of that. Moreover, you can change to zero based if you wish.

So this "very big elephant" is, in reality, a nothingburger.

For me, the very big elephant in the room is the semantic formatting. It has and continues to trip me up time and again. A touch of a space bar can change the flow of a python program. That is a huge asteroid of a problem.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#245

Post by hpcjoe »

jakobnissen wrote: post #164
laichzeit0 wrote: post #148

Earlier quoted context omitted.

Let’s not ignore the giant elephant in the room: 1-based indexing. I don’t particularly care since I use R and Python but Java, C, C/C++, C# all used 0-based indexing. It’s truly a bizarre choice Julia made there.

It really is a rather small elephant. It can be jarring at first (unless you think of R, MatLab and Bash), but then you just stop thinking about it because it legit doesn't matter. People should stop wasting time bikeshedding this insignificant detail, but for some reason it is to programmers like a red rag to a bull.

Really its only an issue to some programmers, who prefer specific languages.

When you have to deal with a wide range of languages, stuff like this is small potatoes, compared to, say, indentation based structure. The latter can result in the completely non-obvious change of program flow due to a single errant space or tab.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#246

Post by hpcjoe »

goerz wrote: post #194
dash2 wrote: post #170

Earlier quoted context omitted.

The reason beginners have a hard time with 0 based indexing is that humans count from 1 . Seriously, I've spent weeks trying to tell people "yeah, we want rows 4 and 5, so that's, uh, rows 3 and 4..." and they think it's nuts, and I now think they're right.

Right. Zero based indexing makes zero sense, unless you explain the underlying technical reason, that it’s an offset relative to a memory pointer (spend a week teaching pointers first!). It makes sense in certain context (and in languages like C that have a low-level mental model). For scientific computing at a higher level of abstraction where the mental model of a multidimensional array is a tensor, and not a memor…

Precisely. Indexing makes sense in a context, and it is trivial in general to switch. This said, telling people that the first element they want is the "0th", is completely unnatural.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#247

Post by hpcjoe »

cookieater wrote: post #224
freemint wrote: post #196

Earlier quoted context omitted.

You can turn any 1 based array into a 0 based array using a wrapper type that just gets inlined away. Also have a look at https://github.com/giordano/StarWarsArrays.jl

Mose Giordano hit the nail on the head with this one. Who cares about 1 vs 0 based indices? If anything its a silly and minor design decision. I worry about someones ability to solve real problems in any language if they can't get their head around an +1/-1 when indexing into an array.

> I worry about someones ability to solve real problems in any language if they can't get their head around an +1/-1 when indexing into an array.

This. Think about what this signals to employers and interviewers if someone throws a hissyfit over this.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#248

Post by mrtweetyhack »

edgyquant wrote: post #42
zz865 wrote: post #14

Earlier quoted context omitted.

Wow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too. I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.

I do like JS quite a bit, I was a hater until a few years ago when I had to start working with it more than a snippet here and there, but I still prefer python on the backend. I think JS makes a lot of sense for frontend with the way everything is async.

But if you can use just one why bother learning two?

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#249

Post by mrtweetyhack »

fault1 wrote: post #239
anothernewdude wrote: post #185

Earlier quoted context omitted.

If Julia had wanted to be taken seriously then it shouldn't have dropped the ball at the very first hurdle by having 1-based array indexing.

Julia (and Fortran) have a concept called offset arrays where you can basically start on any sort of index: https://github.com/JuliaArrays/OffsetArrays.jl IMHO, one of the biggest advantages of Julia _is_ arrays.

then why not start at 0 by default?

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#250

Post by agumonkey »

freemint wrote: post #199
ChrisRackauckas wrote: post #195

Earlier quoted context omitted.

You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.

In certain common lisp deployment you might do that. There are legends from Common Lisp users where this is very much done. > When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix…

most famous one being NASA JPL remote patch
Post reply on HN