Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

dev-discuss.pytorch.org

Post reply on HN
Print view View original

191–200 of 291 posts

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#191

Post by krastanov »

creata wrote: post #154
jhgb wrote: post #150

Earlier quoted context omitted.

They're not "essentially equivalent". A while loop can't begin in one module and end in another. A tail call sequence can. Loops are not modular.

Just to be clear, are you suggesting mutual recursion across modules ?

Why is that unreasonable? I might be missing something (I have no formal CS education), but I can imagine it, e.g. a base graph datastructure library, and then two libraries dependent on the base library being used together for some graph traversal work (where callbacks are used).

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#192

Post by freemint »

amelius wrote: post #78
edgyquant wrote: post #44

Earlier quoted context omitted.

I like what Julia is doing but I just dislike the syntax. It seems to resemble ruby, whose syntax I also think is ugly, which to me resembles a modern form of basic.

I know what you mean. I especially dislike the use of an "end" keyword everywhere without a corresponding "begin" keyword.

begin is a Julia keyword though it.

But yes end closes begin, for, function and while.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#193

Post by ChrisRackauckas »

joconde wrote: post #183
mbeex wrote: post #179

Earlier quoted context omitted.

Indeed it is (slight improvements not excluded). I would love to be able to write things like M T A (w/o the HTML rubbish seen here - flavour of the same problem) for two matrices directly as code (can Julia do that?). Instead, I've been forced to learn row and column precedence in OpenGL, DirectX, Numpy, and whatever, paying attention to it in endless constructs. Even after 25 years in the business, it's still a pai…

What about everything that isn't yet represented by widely-adopted mathematical notation? I'd hate to have to learn which Greek letters correspond to which operations every time I dive into a new codebase. The advantage of English words is that everyone knows them. Edit for details: `Σλ∀φ` is a chore to read for me. These letters only make sense if I've absorbed the author's notation beforehand, like in a math proof.…

> `Σλ∀φ` is a chore to read for me. These letters only make sense if I've absorbed the author's notation beforehand, like in a math proof.

It's a hell of a lot better than reading `Sigma`, `lambda`, etc. and having to do both the verbose English -> character translation in your head while trying to understand the mathematical form. At the end of the day, you do need to understand the math in order to understand mathematical code, and making it have as few translation steps really helps.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#194

Post by goerz »

dash2 wrote: post #170
JuettnerDistrib wrote: post #166

Earlier quoted context omitted.

You mean, the little ant on the floor? I barfed at 1-based indexing for about a week, but now it is as natural as anything. I would compare 0-based and 1-based indexing with whether you put semicolons at the end of each line or not. Either way doesn't really change the feel (semantics) of the language. Also, fortan is 1-based, iirc, and a lot of numerical code is in fortan. Oh, and many many beginning programmers and…

The reason beginners have a hard time with 0 based indexing is that humans count from 1 . Seriously, I've spent weeks trying to tell people "yeah, we want rows 4 and 5, so that's, uh, rows 3 and 4..." and they think it's nuts, and I now think they're right.

Right. Zero based indexing makes zero sense, unless you explain the underlying technical reason, that it’s an offset relative to a memory pointer (spend a week teaching pointers first!).

It makes sense in certain context (and in languages like C that have a low-level mental model). For scientific computing at a higher level of abstraction where the mental model of a multidimensional array is a tensor, and not a memory offset location, zero-based indices really get in the way

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#195

Post by ChrisRackauckas »

freemint wrote: post #190
longemen3000 wrote: post #125

Earlier quoted context omitted.

Just to be clear Julia is "AOT-JIT", that means that the methods are compiled when used the first time. With Revise.jl (a package for interactive development) you can make a fully interactive Julia process, rewriting things on the fly, while benefiting from fast code

Running Revise in production sounds unwise (sorry for the pun). Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).

You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#196

Post by freemint »

laichzeit0 wrote: post #148
hpcjoe wrote: post #16

Earlier quoted context omitted.

This is one of the nicer aspects of Julia. It starts out being a great language to work in. Its easy to implement algorithms that are generally difficult in other languages. Its important to remember that most of the python ecosystem, isn't written in python. The functions are often thin wrappers/objects around the real computation, which is often written in a faster language, C/C++/Fortran. Julia excels in composabi…

Let’s not ignore the giant elephant in the room: 1-based indexing. I don’t particularly care since I use R and Python but Java, C, C/C++, C# all used 0-based indexing. It’s truly a bizarre choice Julia made there.

You can turn any 1 based array into a 0 based array using a wrapper type that just gets inlined away.

Also have a look at https://github.com/giordano/StarWarsArrays.jl

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#197

Post by ChrisRackauckas »

anothernewdude wrote: post #185
pilooch wrote: post #182

Loving pytorch but the reasoning remains strange to my ears: Python at all cost, not Julia, because of the ecosystem, well OK. But all bullet points are about things that are easily done right now with libtorch (pytorch underlying C++ core code), and the hassle is... Python. Well rational conclusion would be, just do everything in C++, and bind to Python. Make C++ first citizen here, since in all cases it'll be neede…

If Julia had wanted to be taken seriously then it shouldn't have dropped the ball at the very first hurdle by having 1-based array indexing.

The person who comes to a Julia discussion and says "so, how's 1-based indexing?" is the same person who walks up to people in the office and goes "so, how's the weather today?" every single day. It's the Smash Mouth Allstar of programming language conversations. The only things interesting about the conversation at this point are the ever more elaborate attempts at semi-comedically changing the topic towards something that's not so overly repeated.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#198

Post by freemint »

jhgb wrote: post #153
freemint wrote: post #13

Earlier quoted context omitted.

I've never seen recursive looping over a thing get turned into efficient SIMD code in any language. Starting with loops has no good reason to be better able to achieve that but for practical compilers it makes a huge difference. Julia code might also uses a lot of in place operations which would be hard for a compiler to infer as safe.

> I've never seen recursive looping over a thing get turned into efficient SIMD code in any language. Starting with loops has no good reason to be better able to achieve that but for practical compilers it makes a huge difference. Well, for example at the very least in Common Lisp you'll have much more joy with higher-order functions than with loops. The simple reason for that is the existence of compiler macros ( ht…

Maybe loops are not fun in common lisp the? I had much fun with loops in PicoLisp.

Julia also has similar macro capabilities to common lisp.

> And it's much easier to figure out what the function composition does than to write a loop vectorizer

I probably agree but complicated loop vectorisation would probably expressed in Einstein tensor contraction notation such as Tullio.jl provides. I wonder a Fourier transform would look using function decomposition.

@tullio F[k] := S[x] * exp(-impi/8 (k-1) * x) (k ∈ axes(S,1))

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#199

Post by freemint »

ChrisRackauckas wrote: post #195
freemint wrote: post #190

Earlier quoted context omitted.

Running Revise in production sounds unwise (sorry for the pun). Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).

You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.

In certain common lisp deployment you might do that.

There are legends from Common Lisp users where this is very much done.

> When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone.

> Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun.

Taken from https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=163...

I know that this is not possible in Julia like that. But just because it's not possible in Julia doesn't mean what they did was somehow wrong when it worked well for them.

Re: PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

#200

Post by ChrisRackauckas »

goerz wrote: post #194
dash2 wrote: post #170

Earlier quoted context omitted.

The reason beginners have a hard time with 0 based indexing is that humans count from 1 . Seriously, I've spent weeks trying to tell people "yeah, we want rows 4 and 5, so that's, uh, rows 3 and 4..." and they think it's nuts, and I now think they're right.

Right. Zero based indexing makes zero sense, unless you explain the underlying technical reason, that it’s an offset relative to a memory pointer (spend a week teaching pointers first!). It makes sense in certain context (and in languages like C that have a low-level mental model). For scientific computing at a higher level of abstraction where the mental model of a multidimensional array is a tensor, and not a memor…

> Zero based indexing makes zero sense

The sensibility of the index choice is equal to the starting value of the index.

Post reply on HN