Live data from Hacker News

Automatic Differentiation with Julia

blog.rogerluo.me

21–30 of 83 posts

Re: Automatic Differentiation with Julia

#22
post #11

Earlier quoted context omitted.

> or any other kind of offset that you so desire This is too often the attitude of Julia people. But zero-based indexing is not just some arbitrary offset. Programmers just like it. You say that many things are more intuitive with one-based indexing, but many things are less intuitive too. I don't think python ever would have become popular without zero-based indexing. The thing is, we really don't know why people pr…

> But zero-based indexing is not just some arbitrary offset. Programmers just like it. Um, that's the definition of "arbitrary".

Programmers might like it for no known reason and as such the preference might be arbitrary, but the choice for a language to reflect that preference is not.

Re: Automatic Differentiation with Julia

#23
post #17
post #11

Earlier quoted context omitted.

> or any other kind of offset that you so desire This is too often the attitude of Julia people. But zero-based indexing is not just some arbitrary offset. Programmers just like it. You say that many things are more intuitive with one-based indexing, but many things are less intuitive too. I don't think python ever would have become popular without zero-based indexing. The thing is, we really don't know why people pr…

Well for one, most situations I can think of when an index is calculated a 0-based index is more useful. Addressing an element T[h][w][c] in a linearized 3D tensor with 0-based indexing: T[h*W*C+w*C+c] with 1-based indexing: T[(h-1)*W*C+(w-1)*C+c] Or let's say you want to take a string "abc" and repeat it until the length is 10, getting "abcabcabca". With 0-based indexing: a,b = "abc", [" "]*10 for i in range(0,10):…

One-based indexing obviously means that ranges are inclusive on the right, and you iterate by `for i = 1:length(x)`.

That being said, I also abhor 1-based indexing in julia.

Well, you get used to it, and it is imho a small price to pay for julia's other cool things.

In the end, julia needs to access arrays through pointers, and indirect adressing / LEA use zero-based indexing. Somewhat amazingly, llvm manages to remove/hoist most of the extraneous subtractions. But still, it would be much more transparent if julia simply reflected the hardware standard of zero-based addressing (when in doubt, stay as close to the silicon as reasonably possible).

Offset arrays are a bad choice when avoidable, because (1) they incur an additional pointer load, (2) most julia code uses 1-based indices (conforming to common language idioms is always good), and (3) you will trigger all sorts of bugs in all sorts of packages that falsely assume that all AbstractArray are one-based.

Re: Automatic Differentiation with Julia

#24
post #17
post #11

Earlier quoted context omitted.

> or any other kind of offset that you so desire This is too often the attitude of Julia people. But zero-based indexing is not just some arbitrary offset. Programmers just like it. You say that many things are more intuitive with one-based indexing, but many things are less intuitive too. I don't think python ever would have become popular without zero-based indexing. The thing is, we really don't know why people pr…

Well for one, most situations I can think of when an index is calculated a 0-based index is more useful. Addressing an element T[h][w][c] in a linearized 3D tensor with 0-based indexing: T[h*W*C+w*C+c] with 1-based indexing: T[(h-1)*W*C+(w-1)*C+c] Or let's say you want to take a string "abc" and repeat it until the length is 10, getting "abcabcabca". With 0-based indexing: a,b = "abc", [" "]*10 for i in range(0,10):…

Some algorithms work better with 0-based indexing. But maybe, just maybe, there are other aspects to look at when evaluating a programming language than "it doesn't by default follow my favourite indexing type". Especially given that 0-based indexing is very easy to use for any array you may wish to use it for.

Re: Automatic Differentiation with Julia

#25
post #17

Earlier quoted context omitted.

Well for one, most situations I can think of when an index is calculated a 0-based index is more useful. Addressing an element T[h][w][c] in a linearized 3D tensor with 0-based indexing: T[h*W*C+w*C+c] with 1-based indexing: T[(h-1)*W*C+(w-1)*C+c] Or let's say you want to take a string "abc" and repeat it until the length is 10, getting "abcabcabca". With 0-based indexing: a,b = "abc", [" "]*10 for i in range(0,10):…

One-based indexing obviously means that ranges are inclusive on the right, and you iterate by `for i = 1:length(x)`. That being said, I also abhor 1-based indexing in julia. Well, you get used to it, and it is imho a small price to pay for julia's other cool things. In the end, julia needs to access arrays through pointers, and indirect adressing / LEA use zero-based indexing. Somewhat amazingly, llvm manages to remo…

In fact, almost every solid julia package uses arbitrary indexing for input arrays. `for i = 1:length(x)` is discouraged in production code, instead the ideom is `for i in eachindex(x)`.

Re: Automatic Differentiation with Julia

#26
post #14
post #10

Earlier quoted context omitted.

And they already have Julia! Looks like you can't joke with array indexing...

Actually there are plenty of options with Algol derived languages.

Apropos, as Algol allowed a user-defined base type, with the default as 1. https://en.wikipedia.org/wiki/Comparison_of_programming_lang...

Re: Automatic Differentiation with Julia

#27
post #17
post #11

Earlier quoted context omitted.

> or any other kind of offset that you so desire This is too often the attitude of Julia people. But zero-based indexing is not just some arbitrary offset. Programmers just like it. You say that many things are more intuitive with one-based indexing, but many things are less intuitive too. I don't think python ever would have become popular without zero-based indexing. The thing is, we really don't know why people pr…

Well for one, most situations I can think of when an index is calculated a 0-based index is more useful. Addressing an element T[h][w][c] in a linearized 3D tensor with 0-based indexing: T[h*W*C+w*C+c] with 1-based indexing: T[(h-1)*W*C+(w-1)*C+c] Or let's say you want to take a string "abc" and repeat it until the length is 10, getting "abcabcabca". With 0-based indexing: a,b = "abc", [" "]*10 for i in range(0,10):…

In Julia, you don't have to calculate linear indices yourself since you can create a reshaped view of a blob of memory in arbitrary number of dimensions. So this case doesn't come up. Also there are plenty of cases where 0-indexing is more awkward than 1-indexing. Actually reading your Dijkstra link will make that pretty clear. Reading it will also make it clear that the 0-indexing choice is pretty arbitrary and that you can make a very similar case for 1-indexing too. This 0-indexing meme really needs to die, especially the argument from authority by linking to Dijkstra.

Re: Automatic Differentiation with Julia

#28
For those unaware of what automatic differentiation is: It's a close-to-magical tool which turns code for evaluating a function f into code for evaluating its derivative f'.

It uses a special sort of invented numbers which square to zero even though they are not themselves zero.

Here is one of the many tutorials on automatic differentiation: https://pizzaseminar.speicherleck.de/automatic-differentiati...

Re: Automatic Differentiation with Julia

#29
post #27
post #17

Earlier quoted context omitted.

Well for one, most situations I can think of when an index is calculated a 0-based index is more useful. Addressing an element T[h][w][c] in a linearized 3D tensor with 0-based indexing: T[h*W*C+w*C+c] with 1-based indexing: T[(h-1)*W*C+(w-1)*C+c] Or let's say you want to take a string "abc" and repeat it until the length is 10, getting "abcabcabca". With 0-based indexing: a,b = "abc", [" "]*10 for i in range(0,10):…

In Julia, you don't have to calculate linear indices yourself since you can create a reshaped view of a blob of memory in arbitrary number of dimensions. So this case doesn't come up. Also there are plenty of cases where 0-indexing is more awkward than 1-indexing. Actually reading your Dijkstra link will make that pretty clear. Reading it will also make it clear that the 0-indexing choice is pretty arbitrary and that…

Contrary to your summary, the quote makes a good case for 0-based indexing, both theoretical and practical. How can an argument based on reason become an argument from authority just because someone well known said it?

Re: Automatic Differentiation with Julia

#30
post #11

Earlier quoted context omitted.

So, fun times: if the 1-based indexing throws you off that much, it is entirely straightforward to configure it to use 0 based indexing if you want (or any other kind of offset that you so desire) https://docs.julialang.org/en/latest/devdocs/offset-arrays/ Having said that, I encourage you to try out the 1-based indexing, as I think you might find a lot of things become surprisingly more intuitive.

> or any other kind of offset that you so desire This is too often the attitude of Julia people. But zero-based indexing is not just some arbitrary offset. Programmers just like it. You say that many things are more intuitive with one-based indexing, but many things are less intuitive too. I don't think python ever would have become popular without zero-based indexing. The thing is, we really don't know why people pr…

[deleted]
Post reply on HN