Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

341–350 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#341

Earlier quoted context omitted.

Kids these days, eh? Lawn, etc.

Nah, it's not like that. In my mind an "older" programmer is like from the 90s. But I did that stuff in the mid-2000s too: am I now an "old"? Terrifying!

Relax: No, you're not. You've got another fifteen years.

Re: Correctness and composability bugs in the Julia ecosystem

#342

Earlier quoted context omitted.

> What index to start with only strongly matters when the indexes have semantics. Which in everyday computing (as opposed to mathematics) they often do, and those cases are (most?) often, in human terms, much more natural to start from 1: "I have an array of N elements. The first of a bunch of things is thing number one, and the last of N things is thing number N." Hence: N_things: Array[1-N] of Thing; for i := 1 to…

Algol had negative indexes. You could declare an array of nine elements going from -4 to 4, for example. I couldn't find why they wanted such a thing.

Hm, maybe Pascal does too. Can't recall.

Re: Correctness and composability bugs in the Julia ecosystem

#343

Earlier quoted context omitted.

> 2) you can find out what methods you need to implement just by running the code that uses the implementation and see what fails. For large codebases this is SO painful to do. I just don't understand how anyone gets anything done when this is how they have to develop code.

That's why interfaces are useful—they save you from that. But they don't actually solve the problem of checking that an abstraction has been implemented correctly, just that you've implemented the entire API surface area, possibly incorrectly. Note, however, that if you have a way of automatically testing the behavioral correctness of an implementation, then those tests presumably cover the entire API, so automatic t…

I think I agree with your claim that writing tests would be a super set in terms of covering the use cases of interfaces. But

1) Testing is such a PAIN in Julia. You HAVE to run all the tests every time. You HAVE to write tests in a separate folder. Multiple dispatch and composibility prevent having confidence that your tests cover all the cases.

2) In a lot of cases, interfaces improve readability of the code. Being able to just look at code in a large code base and know which interfaces are implemented + need to be implemented is such an advantage

3) Static analysis tooling can provide linting. This doesn't even have to be implemented by the core team at the moment. The lack of interface limits any kind of tooling to be developed.

All in all, when I write Julia code, I know I have to write EXTENSIVE tests. Even more tests than I have to write with Python (with mypy). Almost an order of magnitude more tests than I have to write with Rust.

Sometimes it feels like I spend more time writing / running tests than adding features to our packages. And that is honestly just not fun for me, let alone for other scientists and researchers on my team.

Re: Correctness and composability bugs in the Julia ecosystem

#344

Earlier quoted context omitted.

`eachindex` is — in quite a few situations — faster than `1:n`. We've also been trying to promote a culture of not blindly putting `@inbounds` notations on things as the compiler gets smarter. `@inbounds` is a hack around a dumb compiler, especially when the loop is as simple as many of these examples. It's not needed there anymore (but was 5 years ago).

Perhaps that is part of the point of the article? If you accept things like @inbounds, which is a horrible hack and was a horrible hack five years ago, then perhaps the culture is a little too tolerant towards horrible hacks. Because many of the bugs the author enumerates are of the "fixes the problem for now, let's deal with the consequences later" type.

I wouldn't say that @inbounds is a "horrible hack". Just like the `unsafe` part of Rust is not a "horrible hack". There are cases where it is impossible for a compiler to statically verify that an index access is in bounds and in those cases it will need to emit a check and an exception. This prevents many other optimizations (for example SIMD). So for a language that is intended for people to write low-level numerical routines there has to be a way to opt out of these checks or people would have to write their numerical routines in a completely different language. But the important part is that index access is memory safe by default (as opposed to e.g. C) and you can also force boundschecking to be turned on (to override @inbounds) with a command-line flag (--check-bounds=yes). So if you want, you could pretend "@inbounds" doesn't exist by just aliasing your julia executable to apply that command line flag.

Re: Correctness and composability bugs in the Julia ecosystem

#345
post #316

Earlier quoted context omitted.

FWIW my take is not that Yuri is expressing "there are too many bugs" so much as he's expressing a problem in the culture surrounding Julia itself: > But systemic problems like this can rarely be solved from the bottom up, and my sense is that the project leadership does not agree that there is a serious correctness problem. Concisely: 1. The ecosystem is poorly put together. (It's been produced by academics rather t…

> The Julia language is amazing. The ecosystem needs to be rewritten. I think this is pretty unfair. Julia has many libraries that have allowed me to build things that would have taken orders of magnitude more effort to produce in other languages with the same conciseness and efficiency. Composability and efficiency hard. Are things better elsewhere? Python has excellent libraries. But these are big monoliths that no…

> Are things better elsewhere? Python has excellent libraries. But these are big monoliths that not only do not compose well, but are also hard to understand deeply as they are essentially a thin layer over C, C++, Fortran, etc.

I dunno.

Things like the use of scipy.spatial.distance metrics[1] by in sklearn clustering[2] seems a great example of composability that is easy to learn and very efficient.

And the sklearrn side isn't a "thing layer over C, C++, Fortran" even if scikit is (sort of) this.

[1] https://docs.scipy.org/doc/scipy/reference/spatial.distance....

[2] https://scikit-learn.org/stable/modules/generated/sklearn.me...

Re: Correctness and composability bugs in the Julia ecosystem

#346
post #326

Earlier quoted context omitted.

Lots of things are being rewritten. Remember we just released a new neural network library the other day, SimpleChains.jl, and showed that it gave about a 10x speed improvement on modern CPUs with multithreading enabled vs Jax Equinox (and 22x when AVX-512 is enabled) for smaller neural network and matrix-vector types of cases ( https://julialang.org/blog/2022/04/simple-chains/ ). Then there's Lux.jl fixing some majo…

The fact that things are being rewritten and the primary criteria being looked at is speed IS culturally a big part of the problem. If you don't prioritize provable correctness first, then I guarantee that the code is not correct. And as the complaint explains, incorrect code costs people months and leads them to not trust the result. Don't believe me? Re-read the blog post about how a major source of bugs is people…

> Re-read the blog post about how a major source of bugs is people making assumptions into silent errors by removing bounds checks. Simply being able to re-run the same code in a slow mode with the bounds checks turned back on would undoubtably catch bugs.

Running Julia with the command line argument --check-bounds=yes does that, and package testing always uses this option to disable inbounds.

Re: Correctness and composability bugs in the Julia ecosystem

#347
post #224

Earlier quoted context omitted.

"known to be incompatible" Known to whom? People who regularly participate in the Julia forum/chat? Julia's composability relies on people agreeing on unwritten rules and standards. In other languages, such incompatibilities are caught by the compiler. Even in other dynamic languages like Python or Javascript, it is now considered best practice by many to annotate types whenever you can. Like Julia, Haskell is also c…

Agreed, one cannot just expect this to be known. Does type annotations in Python actually catch type errors? I thought they were mainly for documentation.

Absolutely yes, but you have to use a typechecker like mypy (and generally make it part of your release builds). I've found typechecking my Python code makes my development iterations much faster than writing tests. My biggest issue is that if you are using a legacy codebase or 3P library without type annotations then the "Any" type become pervasive and removes much of the value you get from type annotations. You can run mypy in a mode that flags when this is happening, but it's not like you're going to go type annotate the world just to push your code change.

Re: Correctness and composability bugs in the Julia ecosystem

#348

Earlier quoted context omitted.

Yeah because humans got it wrong. Really the word for "first" should correspond to the number 0. Try doing a block iteration over an array, or any kind of interval algorithms in 0-based and 1-based. 0-based with right-open intervals just results in way way more elegant, easier to understand and (very) slightly more efficient code.

The reason why this 0 vs 1 based indexing debate is never resolved is because all of the arguments are subjective. You've claimed definitively that "humans got it wrong", but to back up this argument you've pointed to vague notions of "elegance" and "understandability". Even Dijkstra in his argument relies on a notion of "ugliness". All such arguments fall squarely in the realm of "preferences". Just think about what…

There are non-subjective arguments. See EWD831: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...

Re: Correctness and composability bugs in the Julia ecosystem

#349

Earlier quoted context omitted.

> Everything has correctness issues somewhere. This is fallacy of gray. The blog post isn't complaining that there are non-zero bugs, it's complaining that when you use the language you hit a lot of correctness bugs. More bugs than you'd hit using e.g. python. Also, to the extent that Julia uses LLVM, a correctness bug in LLVM is also a correctness bug in Julia. So arguing "LLVM has lots of correctness bugs" is not h…

> So arguing "LLVM has lots of correctness bugs" is not helping the case It does not help the case about the correctness of Julia, but it does help the case about Julia having more bugs than other software (negatively for the other projects). Every library built with LLVM that touches those code paths will have those bugs. Another thing to have in mind is that Julia ships patches for some of these, that are not used…

It shows that Julia's tests are systematically finding (and leading to fixes) of numerical bugs that are pervasive throughout the rest of the LLVM ecosystem. And since Julia's LLVM is patched to solve these while other variants of LLVM are not, Julia is more correct in these aspects than other languages which rely on the Base build of LLVM. Of course Julia doesn't solve "all bugs", but some of them (like the correctness of certain math library implementations) really make you question how hard other language tests are hammering those for correctness testing (Julia has a lot of numerical tests checking the precision of such methods against MPFR bigfloats at higher precision to ensure ~X ulp correctness for example). Julia definitely spends a lot of time testing numerical correctness than it does testing something like a web server. It's just a prioritization thing.

Re: Correctness and composability bugs in the Julia ecosystem

#350
post #221
post #169

Earlier quoted context omitted.

GP was addressing the argument. Using the word "you" in the reply does not make it ad hominem. (For more than you ever wanted to know about this, The Ad Hominem Fallacy Fallacy covers it exhaustively and entertainingly: https://laurencetennant.com/bonds/adhominem.html )

This is an interesting point, though, respectfully, I do still think it's ad homenim. Internet arguments being what they are I don't much care, but I offer my reasoning here to better understand your point. OP did not engage with any of the points made, merely offering another term (without any sort of elaboration or definition), and said > Notice how you had to put index in quotes. as the thrust of the argument. In…

>OP did not engage with any of the points made, merely offering another term (without any sort of elaboration or definition), and said Notice how you had to put index in quotes.

Yes. That's addressing the point you made.

Ad hominem would be: "You're a bad person/you have this or that flaw/etc (unrelated personal stuff)".

This is: "You put the index in quotes, because even you know that this is not an index. And in any case, this is not considered an index in math, it's a degree, which is a different thing".

I also didn't "merely offered another term", as if I made up some term on my own, or just offered on of several equal alternatives. Instead, I gave the correct math term for the thing described.

>In saying that, they imply that I, the arguer, 'A' in Bond's article, don't actually know what an index is (so how could I have a cogent argument about 'correct' indexing?).

It would rather imply the opposite: that you know what an index is, and you know that the thing you applied it to, is not an index (which is why it was put in scare quotes).

Post reply on HN