Correctness and composability bugs in the Julia ecosystem • Hacker News Board

Re: Correctness and composability bugs in the Julia ecosystem

#91

Post by ThenAsNow » Mon, May 16, 2022, 2:58 PM UTC

QuackingTheQ wrote: post #52

I've spent a lot of time developing large computational codebases in Julia, and I think the most insidious of these issues is a product of no formal way of enforcing interfaces. Using one of the common packages to build a trait system and add some sort of guarantee that all the right methods are implemented for a given trait simplifies maintenance dramatically. This doesn't catch mathematical bugs, but those crop up…

I agree with the kernel of your point here, but also with the author of the article when he says "But systemic problems like this can rarely be solved from the bottom up, and my sense is that the project leadership does not agree that there is a serious correctness problem. They accept the existence of individual isolated issues, but not the pattern that those issues imply."

My impression is that the Julia core devs are more focused on functionality and being able to construct new, more powerful, faster capabilities than on reflecting on how the foundations could or should be made more rigorous. For this, I think the devs have to philosophically agree that soundness in the large should be a first-tier guiding principle, and that the language should have mechanisms whereby correctness-by-construction can be encouraged, if not enforced. Presently, notions of soundness seems to only be considered in the small, such as the behavior of specific floating point ops. Basically, I don't think the core devs are as concerned with soundness, rigor, and consistency as they are with being able to build more impressive capabilities.

I don't want this to sound like I'm ungrateful for the awesomeness that Julia and its ecosystem does bring to the table. For numerical computing, I don't see any alternatives whose tradeoffs are more favorable. But it is disappointing that it doesn't seem to learn the lessons about rigorous language design and the language-level implications for engineering vs. craftsmanship appropriate for a twenty-first century language.

Re: Correctness and composability bugs in the Julia ecosystem

#92

Post by krastanov » Mon, May 16, 2022, 2:59 PM UTC

Gwypaas wrote: post #76

dandanua wrote: post #54

Earlier quoted context omitted.

Julia has more than 18k closed issues on its github. No wonder such an active user encountered a lot of it. It's not a problem with the language, though. Yes, it allows to use offsetarrays and @inbounds together, but C can read out-of-memory locations too, so what? Edit: Julia is better than C in this regard, since the usage of @inbounds is explicit, i.e. everyone can see that the code is potentially unsafe.

> but C can read out-of-memory locations too, so what? Simply decades of exploitable security issues.

The Julia example is closer to Rust's `unsafe`. Pretty much every language let's you skip bound checks, in Julia (like other modern languages) it is elective. The author was complaining about a library that decided to skip the bound check in a clumsy way (there happens to be a "correct" way to skip the bound check). It is not really about the language.

Re: Correctness and composability bugs in the Julia ecosystem

#93

Post by masklinn » Mon, May 16, 2022, 3:00 PM UTC

dandanua wrote: post #54

xt00 wrote: post #9

If you look at the history of lots of packages in matlab they fixed tons of bugs that sound similar to this stuff over the years. It requires consistent hard work by a core group of people who understand the issues to get everything right. I have no idea who maintains Julia and these packages but the author of the article mentions this as language problems — aren’t these just bugs? Like if gcc was incorrectly multipl…

Julia has more than 18k closed issues on its github. No wonder such an active user encountered a lot of it. It's not a problem with the language, though. Yes, it allows to use offsetarrays and @inbounds together, but C can read out-of-memory locations too, so what? Edit: Julia is better than C in this regard, since the usage of @inbounds is explicit, i.e. everyone can see that the code is potentially unsafe.

> but C can read out-of-memory locations too, so what?

So it's widely considered a plague upon the field, suffered because of the lack of alternative?

Re: Correctness and composability bugs in the Julia ecosystem

#94

Post by orbifold » Mon, May 16, 2022, 3:01 PM UTC

dekhn wrote: post #4

Wait, are those examples real? I remember complaining about 1-bsaed indexing only to be told "julia is great! we have offsetindex". If it's a source of bugs, that ... greatly reduces my future interest in adopting the language.

I was bit by trying to figure out of how to combine unit of measurements with other numerical computations. Ultimately a lot of the features look great on paper, but once you start using them, I only ever was able to produce an ungodly mess instead of what I could accomplish in Python in roughly the same time. Everything that goes beyond what Matlab does, sometimes looks great on paper but is not very pleasant to use / sometimes badly broken unfortunately. That being said I work in an area of scientific research where Julia or more specifically DifferentialEquations.jl would seem to walk away with the win, but I find myself searching for alternatives implemented in Jax.

I would still think most of this is my failings, but it is also extraordinarily hard to figure out what is going wrong.

Re: Correctness and composability bugs in the Julia ecosystem

#95

Post by tylermw » Mon, May 16, 2022, 3:01 PM UTC

fluidcruft wrote: post #58

s_Hogg wrote: post #12

Earlier quoted context omitted.

It seems like the point of the article is that that push is insubstantial, if it even exists. Given the language has been around this long it's a bit worrying that stuff like that is the potential target for moving a language forward. Julia has always had a reputation in my mind at least of being "by academics, for academics" and there's unfortunately a dark side to that in terms of reliability and maintainability. T…

For what it's worth many people feel similarly about R. R is great for people actively working in statistics research (I assume because that's what I'm always told). But for a lot of us who just want to do some analysis, it's constantly breaking and we've learned to default to just starting from scratch when we need to revisit something we did a few years ago. Or we figure out how to buy a commercial system.

R is not constantly breaking. R Core does a remarkable job ensuring backwards compatibility. There are only a few prominent examples of significant "breaking" behavior across decades of the language existing, and those can often be reverted by setting an option (e.g. `options(stringsAsFactors = TRUE)`). But backwards compatibility is the primary concern with any update to the R language or the packages maintained by R Core.

Now, if you're thinking about changes introduced by a specific user-contributed package breaking your analysis, that can indeed be a problem. But that can't be blamed on the R language. And the main user-contributed R statistics packages that have been around for decades (such as lme4 or survival) are mature and stable.

Re: Correctness and composability bugs in the Julia ecosystem

#96

Post by lahvak » Mon, May 16, 2022, 3:03 PM UTC

karmakaze wrote: post #11

dekhn wrote: post #4

Wait, are those examples real? I remember complaining about 1-bsaed indexing only to be told "julia is great! we have offsetindex". If it's a source of bugs, that ... greatly reduces my future interest in adopting the language.

I was wondering if the 1-based arrays (and option to change index base) would factor into this. > OffsetArrays in particular proved to be a strong source of correctness bugs. The package provides an array type that leverages Julia’s flexible custom indices feature to create arrays whose indices don’t have to start at zero or one. Array indexing is such a core thing and I don't understand why anything mathematical or…

As other posters noted, in mathematics both 0 based and 1 based indexing is used.

When dealing with matrices and vectors (including data tables and data columns), there is a strong preference for 1 based indexing: first row, first column, first entry, etc. Most matrix and vector based algorithms in literature use 1 based indexing. Programming these in a language with 0 based indexing is a mess, and a common source or errors.

When dealing with sequences, especially recursively defined ones, there is usually an initial value (indexed with 0) and then the n-th value is obtained by n applications of the recursive step, so 0 based indexing makes more sense, but in literature there is no fixed convention, and you can find examples with 0 based and with 1-based indexing. Another example of 0 based indexing in math are polynomials (and in extension, power series) where the index is the degree of the term, or in general any functional series where the 0-th term is the constant term.

There are also negative indices.

Re: Correctness and composability bugs in the Julia ecosystem

#97

Post by mattkrause » Mon, May 16, 2022, 3:04 PM UTC

RcouF1uZ4gsC wrote: post #17

> If you pass it an array with an unusual index range, it will access out-of-bounds memory: the array access was annotated with @inbounds, which removed the bounds check. It think making indexes configurable is a huge mistake. Even if they are not ideal for the situation, having a single way to do indexes makes a huge source of confusion and potential bugs just go away. And this is orthogonal to whether you pick 0 or…

There's a parallel idea, that you should avoid--insofar as is possible--numerical indexing. In other words, instead of iterating over `0:length(X) - 1` or `1:length(X)`, you use something like `for element in array` or

    indices = CartesianIndices(multidimensional_X)
    for index in indices

       X[index] = # whatever

If you do that, you don't need to keep track of whether it's zero-based, one-based, or anything else. In fact, you may not even need to keep track of the number of dimensions, as in this example, https://julialang.org/blog/2016/02/iteration/

Re: Correctness and composability bugs in the Julia ecosystem

#98

Post by throwawaymaths » Mon, May 16, 2022, 3:05 PM UTC

temp8964 wrote: post #32

karmakaze wrote: post #11

Earlier quoted context omitted.

I was wondering if the 1-based arrays (and option to change index base) would factor into this. > OffsetArrays in particular proved to be a strong source of correctness bugs. The package provides an array type that leverages Julia’s flexible custom indices feature to create arrays whose indices don’t have to start at zero or one. Array indexing is such a core thing and I don't understand why anything mathematical or…

> Array indexing is such a core thing and I don't understand why anything mathematical or scientific would start with 1. From data analytic point of view, indexing should start with 1. When we analyze a data table, we always call the first row as the 1st row, or row #1, not row #0. It will be very strange to label rows as 0, 1, 2, 3, .... It may be fine for people with Computer Science background. But it would create…

It causes problems for people with a CS background too. I once numbered machines in racks with zero-indexing (so that they could match up with zero-indexed ip addresses). Even though literally everyone who touched those machines had CS background: DO NOT DO THIS.

Re: Correctness and composability bugs in the Julia ecosystem

#99

Post by chalst » Mon, May 16, 2022, 3:07 PM UTC

SemanticStrengh wrote: post #77

one-more-minute wrote: post #62

It might be useful to separate the issues that are "just" bugs from the problems that come with Julia's unusual level of composability. I have no idea if Julia has more bog-standard, local bugs – things like data structure problems or compiler faults – than other languages of comparable maturity and resources, but clearly the OP has bumped into several, which is frustrating. The composition bugs – as in offsetarrays…

any tutorial/blog on what make julia compositionability special vs othe languages? Is there a relation with multiple dispatch or delegation?

Julia has a very nice type system, the nicest of any dynamically typed language I am familiar with. This is something to do with multiple dispatch, but it's more to do with trying to have a type system that allows all the JIT to unbox all the things that have to be unboxed for high performance without sacrificing the freedom of dynamic typing.

IIUC, Common Lisp is the giant on whose shoulders Julia built in this respect.

Re: Correctness and composability bugs in the Julia ecosystem

#100

Post by jacobolus » Mon, May 16, 2022, 3:08 PM UTC

coldtea wrote: post #40

karmakaze wrote: post #11

Earlier quoted context omitted.

I was wondering if the 1-based arrays (and option to change index base) would factor into this. > OffsetArrays in particular proved to be a strong source of correctness bugs. The package provides an array type that leverages Julia’s flexible custom indices feature to create arrays whose indices don’t have to start at zero or one. Array indexing is such a core thing and I don't understand why anything mathematical or…

> Array indexing is such a core thing and I don't understand why anything mathematical or scientific would start with 1. Because starting with 0 is neither math nor array indexing in general. It's just how the base addresss of an array pointer memory block was referenced in C (and it spread from there). Which is why all math focused languages use 1-based (fortran, apl, matlab, r, mathematica, etc.)

Math (usually) uses 1-based indexes because those parts of math started before the concept of zero as a number, and then the convention persisted, even down to Matlab.

There are many similar path-dependent conventions in human culture. E.g. percentages originated before the concept of decimal fractions, base-sixty time units come from ancient Mesopotamia, and conventions about multi-dimensional array memory layout are based on the convention for drawing matrices on paper.

Most common mathematical sequences and series work better (more naturally/clearly) when zero indexing is used instead, and off-by-1 errors are a problem in mathematics just like computing (but less of a problem, because notation errors get silently corrected in readers’ heads, and don’t actually have to be interpreted strictly).