Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

Post reply on HN
Print view View original

71–80 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#71

Post by coldtea »

Rayhem wrote: post #59
coldtea wrote: post #40

Earlier quoted context omitted.

> Array indexing is such a core thing and I don't understand why anything mathematical or scientific would start with 1. Because starting with 0 is neither math nor array indexing in general. It's just how the base addresss of an array pointer memory block was referenced in C (and it spread from there). Which is why all math focused languages use 1-based (fortran, apl, matlab, r, mathematica, etc.)

> Because starting with 0 is neither math nor array indexing in general. It very, very much is. Polynomials all start at a zero "index", as does just about every expansion I can think of (Fourier, Bessel, Legendre, Chebyshev, Spherical Harmonic, etc.) Combinatorics, too, make lots of use of zero indices and zero-sized sets. As for arrays, I'll leave it to Dijkstra[1] to explain why zero indexing is most natural. Zero…

>It very, very much is. Polynomials all start at a zero "index"

Notice how you had to put index in quotes.

Because it's not an index, it's the degree of each polynomial term, which is a power.

Re: Correctness and composability bugs in the Julia ecosystem

#72

Post by ninjin »

I am a long-time member of the Julia community and had a discussion with the author about these issues a long time ago – but did not give feedback on the post. Let me first state that Yuri is a great person and was a valuable member of the community. He pushed the boundaries of the language and produced some very nice packages in his time. His concerns are genuine and should be respected and discussed in that context.

Also, let me say that encountering these kinds of bugs is not something I have had experience with. But, I tend to be very conservative with my usage of libraries and fancy composition.

If I had more experience with programming language theory and implementation, perhaps I would have a better name to describe the source of the issues described. My attempt is to call it “type anarchy”. The way I see it, there is not a clear way to assign responsibility for correctness. In the case of the array used in the post, is it the fault of the implementer of the `sum` function (without a type signature, as it should be) or implementer of the data structure? I am honestly not sure. But as Julia breaks news ground with its type system and multiple dispatch, this could very much be an open question.

Re: Correctness and composability bugs in the Julia ecosystem

#73

Post by one-more-minute »

Q6T46nT668w6i3m wrote: post #25
catchclose8919 wrote: post #10

Only thing "interesting" to me there would be the automatic differentiation bugs ...but is there any argument as to them being the fault of the language, instead of just poor engineering from the library developers' part? I mean, one can't expect all algorithms to work correctly with all datatypes just because the compiler allows that code to run ... you write tests and guarantee numerical stability for a small subse…

I think the author addresses this. It’s a Catch-22. If you restrict use to a small subset of types you’re undermining one of Julia’s best features. As someone who has been writing a lot of numerical analysis code recently, I would absolutely love a type system that could describe and enforce numerical stability traits.

Right. It's important to remember that tools like JAX and PyTorch have total control over the numerical libraries they are differentiating, and have freedom to impose whatever semantics, rules and restrictions are convenient (immutability and referential transparency in JAX, for example). Seemingly small decisions in an existing language and library can have a big impact on the feasibility and practicality of AD.

Re: Correctness and composability bugs in the Julia ecosystem

#74

Post by rob74 »

dandanua wrote: post #54
xt00 wrote: post #9

If you look at the history of lots of packages in matlab they fixed tons of bugs that sound similar to this stuff over the years. It requires consistent hard work by a core group of people who understand the issues to get everything right. I have no idea who maintains Julia and these packages but the author of the article mentions this as language problems — aren’t these just bugs? Like if gcc was incorrectly multipl…

Julia has more than 18k closed issues on its github. No wonder such an active user encountered a lot of it. It's not a problem with the language, though. Yes, it allows to use offsetarrays and @inbounds together, but C can read out-of-memory locations too, so what? Edit: Julia is better than C in this regard, since the usage of @inbounds is explicit, i.e. everyone can see that the code is potentially unsafe.

I think the point he was trying to make was that the example for @inbounds from the official documentation could cause out-of-bounds accesses, while it was clearly stated that you should only use @inbounds if you are sure that no out-of-bounds accesses are possible.

Re: Correctness and composability bugs in the Julia ecosystem

#75

Post by MzxgckZtNqX5i »

jjgreen wrote: post #34
davisoneee wrote: post #27

Earlier quoted context omitted.

I'll start by saying that I greatly prefer 0-based, and have used but 0- and 1-based indexing, but the choice is largely arbitrary. 0 makes sense as the '0-th offset' when thinking from a pointer perspective, but I often find when teaching, that 1-based comes more naturally for many students (the 'first' item). You mention mathematical or scientific work...but I often/mainly see enumerations (such as weights x_1, x_2…

Not for polynomial coefficient indices :-)

Just for consistency: a_n is the coefficient of x^n, so the constant term ends up being a_0. Based on my experience, numbering starts from one (like (x_1, x_2, x_3) as point of R^3) and off-sets from zero, e.g., when dealing with discrete time, t_0 is the first.

Re: Correctness and composability bugs in the Julia ecosystem

#76

Post by Gwypaas »

dandanua wrote: post #54
xt00 wrote: post #9

If you look at the history of lots of packages in matlab they fixed tons of bugs that sound similar to this stuff over the years. It requires consistent hard work by a core group of people who understand the issues to get everything right. I have no idea who maintains Julia and these packages but the author of the article mentions this as language problems — aren’t these just bugs? Like if gcc was incorrectly multipl…

Julia has more than 18k closed issues on its github. No wonder such an active user encountered a lot of it. It's not a problem with the language, though. Yes, it allows to use offsetarrays and @inbounds together, but C can read out-of-memory locations too, so what? Edit: Julia is better than C in this regard, since the usage of @inbounds is explicit, i.e. everyone can see that the code is potentially unsafe.

> but C can read out-of-memory locations too, so what?

Simply decades of exploitable security issues.

Re: Correctness and composability bugs in the Julia ecosystem

#77

Post by SemanticStrengh »

one-more-minute wrote: post #62

It might be useful to separate the issues that are "just" bugs from the problems that come with Julia's unusual level of composability. I have no idea if Julia has more bog-standard, local bugs – things like data structure problems or compiler faults – than other languages of comparable maturity and resources, but clearly the OP has bumped into several, which is frustrating. The composition bugs – as in offsetarrays…

any tutorial/blog on what make julia compositionability special vs othe languages? Is there a relation with multiple dispatch or delegation?

Re: Correctness and composability bugs in the Julia ecosystem

#78

Post by coldtea »

xdavidliu wrote: post #64
coldtea wrote: post #40

Earlier quoted context omitted.

> Array indexing is such a core thing and I don't understand why anything mathematical or scientific would start with 1. Because starting with 0 is neither math nor array indexing in general. It's just how the base addresss of an array pointer memory block was referenced in C (and it spread from there). Which is why all math focused languages use 1-based (fortran, apl, matlab, r, mathematica, etc.)

> It's just how the base addresss of an array pointer memory block was referenced in C (and it spread from there). There was also the famous 1-pager by Dijkstra: "Why numbering should start at zero" https://news.ycombinator.com/item?id=777580

Where the argument was "That is ugly"...

Re: Correctness and composability bugs in the Julia ecosystem

#79

Post by rpmuller »

I've been a part of many language communities, and that the Julia team is the very best in terms of the professionalism of the language and the key modules.

Maybe the best response to this is to view it as a call to action for us Julia fanboys/girls to stop cheering and fix some bugs ;-).

Re: Correctness and composability bugs in the Julia ecosystem

#80

Post by Rayhem »

harpiaharpyja wrote: post #63
tqaky wrote: post #19

Quoted post unavailable.

I'm guessing you've never used Python for a serious project before, because your statement is incorrect. Aside from the fact that Python is memory safe, my experience has been that it is far easier to avoid logic errors in Python than in C/C++. Equivalent Python code is shorter and less noisy than C/C++, and Python makes it easier to create succinct and simple to use abstractions. The result is that code is far easie…

> ...there is a correctness benefit over C/C++ for a disciplined programmer.

Businesses want cheap programmers, not disciplined programmers. It's a great utopia if everyone just follows the best practices and does it perfectly all the time, but I guarantee that as margins become thin people writing, say, code for autonomous vehicles are doing the _very_ minimum to meet correctness requirements.

Post reply on HN