Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

211–220 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#211

Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from tho…

I'll be honest, based on my experience with Julia, this makes me more worried about using e.g. libuv in production systems now, not less. I understand your opinion that "The easier it is to look at the code, the easier it is to find issues with it", but I don't think that has anything to do with the fact that `prod((Int8(100), Int8(100)))` and `prod([Int8(100), Int8(100)])` disagree, because someone decided to specia…

Julia released experimental support for arrays whose indexes don't start at 1 in Julia 0.5, October 2016.

The boundschecking feature was added in 2015, so at the time they wrote their code and examples, they were correct.

The documentation and review happened in December and January 2016/2017 when the non 1-based indexing was still experimental and very new, so I don't think this is as big a fail as you've made out either.

Yes, the documentation should have been updated when non-standard indexing was made non-experimental, and the reviewers should maybe have noted the new and experimental array indexing stuff, but it's only natural to miss some things.

Re: Correctness and composability bugs in the Julia ecosystem

#212
post #192

Earlier quoted context omitted.

Hi Keno, Thanks for the honest assessment. Do you have any thoughts about correctness/ composability of compiler transforms like AD, reliability of GPU acceleration and predictability of optimizations? (basically what you've discussed in some of your compiler talks). How is that going to be possible in an imperative language? Right now we have lux.jl, which is a pure by convention DL framework, but that ends up being…

> Thanks for the honest assessment. What about correctness/ composability of compiler transforms like AD, reliability of GPU acceleration and predictability of optimizations? (basically what you've discussed in some of your compiler talks). I don't think we really have a good answer yet, but it's actively being worked on. That said, I don't think we can be faulted for that one, because I don't think anybody really ha…

Glad to hear it's being worked on!

> That said, I don't think we can be faulted for that one, because I don't think anybody really has a good answer to this particular design problem.

Agreed! To be clear, If there's any implication of "fault" it was certainly not in a moral sense or even anything around making poor design decisions. Julia's compiler is being asked to do many new things with semantics that necessarily predated many advances in PL.

Re Kernel fusion, there's another piece here, which you may or many not have included in "array-level optimizations". Julia's "just write loops" ethos is awesome, until you get to accelerators...now we're back to an "optimizer defined sub language" as TKF puts it. People like loops and flexibility, Dex, Floops.jl, Tullio, Loopvec and KA.jl show that it's possible to retain structure and emit accelerator-able loopy code. But none of those, except for dex, has a solution for fusing kernels that rely on loops. I'm still using the concept of Kernels, because there's still a bit of a separation between low level CUDA.jl code/these various DSLs and higher level array code, even if not as stark as python or C++.

Would be really cool, if like Dex, there's a plan to fuse these sorts of structured loops as well. Dex does it by having type level indexing and loop effects (they're actually moving to a user defined parallel effect handler system (https://arxiv.org/abs/2110.07493) ...the latter can tell the compiler when it's safe to parallelize and fuse+beta reduce loops. But that relies on structured semantics/effects and a higher level IR than exists in Julia.

Not sure what a Julian solution would look like, if possible. But given the usability wins, it would be great to have in Julia as well.

Re: Correctness and composability bugs in the Julia ecosystem

#213
post #208
post #20

Earlier quoted context omitted.

The author mentions that he was stuck on a problem for weeks using Julia, but solved it with Python within hours

That was someone else: Patrick Kidger is mentioned in the article. If I look at the author's github, it's go and javascript.

You're right. I misread.

Re: Correctness and composability bugs in the Julia ecosystem

#214
post #47
post #19

Quoted post unavailable.

If you really want safety, use Rust or Ada.

Or Nim [1]..kind of an Ada with Lisp macros and more Pythonesque surface syntax.

All three are ahead-of-time compiled/less REPL friendly than Julia, though. Taking more than 100 milliseconds to compile can be a deal breaker for some, especially in exploratory data analysis/science settings where the mindset is "try out this idea..no wait, this one..oops, I forgot a -1" and so on. In my experience, it's unfortunately hard to get scientist developers onboard with "wait for this compile" workflows.

[1] https://nim-lang.org/

Re: Correctness and composability bugs in the Julia ecosystem

#215

Earlier quoted context omitted.

I'll be honest, based on my experience with Julia, this makes me more worried about using e.g. libuv in production systems now, not less. I understand your opinion that "The easier it is to look at the code, the easier it is to find issues with it", but I don't think that has anything to do with the fact that `prod((Int8(100), Int8(100)))` and `prod([Int8(100), Int8(100)])` disagree, because someone decided to specia…

Julia released experimental support for arrays whose indexes don't start at 1 in Julia 0.5, October 2016. The boundschecking feature was added in 2015, so at the time they wrote their code and examples, they were correct. The documentation and review happened in December and January 2016/2017 when the non 1-based indexing was still experimental and very new, so I don't think this is as big a fail as you've made out e…

That's fair enough! I was unaware of that history. But my point wasn't that the issue was "a big fail", it's that the GP was unfair in assigning the responsibility of that failure to "some newcomers [who] write code (and documentation) with this" while "the people who really know to handle these cases" are fine. The responsibility should have been on the people pushing for the experimental array indexing code to make it work safely with the existing usage of boundschecks that existed in the ecosystem and the existing documentation. It's a fundamental disagreement between whether the onus of code safety is on the user (who is responsible for understanding the totality of the libraries they're using and all of the ways they can fail) or on the programming language (for ensuring the stability and correctness of its code, documentation and ecosystem when making changes).

Re: Correctness and composability bugs in the Julia ecosystem

#216
post #192

Earlier quoted context omitted.

Hi Keno, Thanks for the honest assessment. Do you have any thoughts about correctness/ composability of compiler transforms like AD, reliability of GPU acceleration and predictability of optimizations? (basically what you've discussed in some of your compiler talks). How is that going to be possible in an imperative language? Right now we have lux.jl, which is a pure by convention DL framework, but that ends up being…

> Thanks for the honest assessment. What about correctness/ composability of compiler transforms like AD, reliability of GPU acceleration and predictability of optimizations? (basically what you've discussed in some of your compiler talks). I don't think we really have a good answer yet, but it's actively being worked on. That said, I don't think we can be faulted for that one, because I don't think anybody really ha…

to add, as you know, this is part of a more general problem about type level programming vs write your own compiler vs the non composability of DSLs, where Julia folks in various other non ML domains like PPLs and fancy arrays have been wondering about how to do things that get compiled away, without relying on compiler heuristics or generated function blowups: https://julialang.zulipchat.com/#narrow/stream/256674-compil...

Another non ML example I discussed with some Probprog folks is that there was an arxiv review of PPLs and Julian ones that heavily rely on macros don't compose well within and across packages. The same mechanism for composability which Dex uses for parallelism and AD (effect handlers) is what new gen PPLs in jax and Haskell are using for composable transformable semantics, so maybe that's worth looking into.

We've been having some discussions about how to bring that to Julia, but stalled on engineering time and PL knowledge. Eventually wanted to talk to the core team about it with proposal in hand, but never got there. Let me know if you'd like to talk to some of those folks who have been involved in the discussions as you design the new compiler plugin infra.

https://julialang.zulipchat.com/#narrow/stream/256674-compil...

Re: Correctness and composability bugs in the Julia ecosystem

#217

> If you pass it an array with an unusual index range, it will access out-of-bounds memory: the array access was annotated with @inbounds, which removed the bounds check. It think making indexes configurable is a huge mistake. Even if they are not ideal for the situation, having a single way to do indexes makes a huge source of confusion and potential bugs just go away. And this is orthogonal to whether you pick 0 or…

There's a parallel idea, that you should avoid--insofar as is possible--numerical indexing. In other words, instead of iterating over `0:length(X) - 1` or `1:length(X)`, you use something like `for element in array` or indices = CartesianIndices(multidimensional_X) for index in indices X[index] = # whatever If you do that, you don't need to keep track of whether it's zero-based, one-based, or anything else. In fact,…

Works great for trivial cases where there's no interdependency between array elements. As soon as you need to access, for example, adjacent elements, you want to be able to just iterate over 1:length(X) - 1 and access a[i-1] and a[i]. This is the most direct way and thus easiest to get right. Abstractions only make it more error prone.

Re: Correctness and composability bugs in the Julia ecosystem

#218
post #214
post #47

Earlier quoted context omitted.

If you really want safety, use Rust or Ada.

Or Nim [1]..kind of an Ada with Lisp macros and more Pythonesque surface syntax. All three are ahead-of-time compiled/less REPL friendly than Julia, though. Taking more than 100 milliseconds to compile can be a deal breaker for some, especially in exploratory data analysis/science settings where the mindset is "try out this idea..no wait, this one..oops, I forgot a -1" and so on. In my experience, it's unfortunately…

Even if you take, 100ms to compile, in practice, it takes about 2 seconds since it requires the user to type in another command and register that it has finished.

Re: Correctness and composability bugs in the Julia ecosystem

#219

Earlier quoted context omitted.

There's a parallel idea, that you should avoid--insofar as is possible--numerical indexing. In other words, instead of iterating over `0:length(X) - 1` or `1:length(X)`, you use something like `for element in array` or indices = CartesianIndices(multidimensional_X) for index in indices X[index] = # whatever If you do that, you don't need to keep track of whether it's zero-based, one-based, or anything else. In fact,…

I'm only skimming this post and I'm not familiar with Julia so maybe I'm missing it, but does it have a way to get an item AND its index? There's I think Enumrable? in Rust where it gives you a tuple with both the item and its index in cases where you need both.

     julia> pairs("François") |> collect                                                                        
     8-element Vector{Pair{Int64, Char}}:                                                            
     1 => 'F'                                                                                                
     2 => 'r'                                                                                                
     3 => 'a'                                                                                           
     4 => 'n'                                                                                                
     5 => 'ç'                                                                                                  
     7 => 'o'                                                                                                  
     8 => 'i'                                                                                                  
     9 => 's'
Notice the missing index 6, because ç takes two bytes.

In contrast, enumerate() gets you the iteration number:

    julia> enumerate("François") |> collect                                                                    
    8-element Vector{Tuple{Int64, Char}}:                                                           
     (1, 'F')                                                                                                
     (2, 'r')                                                                                                
     (3, 'a')                                                                                           
     (4, 'n')                                                                                                
     (5, 'ç')                                                                                                  
     (6, 'o')                                                                                                  
     (7, 'i')                                                                                                  
     (8, 's')
This can trip you up.

Re: Correctness and composability bugs in the Julia ecosystem

#220

Earlier quoted context omitted.

> I agree, code should never do that. It should be `eachindex(A)` Will that generate the same code as "i in 1:length(A)"? Maybe whoever wrote that didn't believe so at least, or perhaps didn't find it so at the time. The reason @inbounds would have been used is performance, so that's likely why the for loop header was written that way?

`eachindex` is — in quite a few situations — faster than `1:n`. We've also been trying to promote a culture of not blindly putting `@inbounds` notations on things as the compiler gets smarter. `@inbounds` is a hack around a dumb compiler, especially when the loop is as simple as many of these examples. It's not needed there anymore (but was 5 years ago).

Perhaps that is part of the point of the article? If you accept things like @inbounds, which is a horrible hack and was a horrible hack five years ago, then perhaps the culture is a little too tolerant towards horrible hacks. Because many of the bugs the author enumerates are of the "fixes the problem for now, let's deal with the consequences later" type.
Post reply on HN