Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem (2022)

yuri.is

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#61

Post by nsajko »

maxbond wrote: post #13

I'm reminded of the recent post about R's (CRAN's) somewhat radical approach to integration testing [1] and I wonder if something like that would help with the composition issues described here. [1] https://news.ycombinator.com/item?id=45259623

The Julia world is already quite careful with testing and CI. Apart from the usual unit testing, many packages do employ integration testing. The Julia project itself (compiler, etc) is tested against the package ecosystem quite often (regularly and for select pull requests).

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#62

Post by nsajko »

hatmatrix wrote: post #41
leephillips wrote: post #39

Earlier quoted context omitted.

This is a reasonable article, but way out of date now. Almost all issues raised were solved a while ago.

I say this as a huge Julia fan, but the point is not the specific bugs in the article, but the culture of not prioritizing correctness in computation. The initial response by many (not all) in the community was look, those specific bugs are fixed; all languages have bugs; more importantly - look at the benchmark speeds of these computations! Which only reinforced this negative perception. My understanding is that it'…

> the culture of not prioritizing correctness in computation

On the contrary, it is my impression the experienced Julia programmers, including those involved in JuliaLang/julia, take correctness seriously. More so than in many other PL communities.

> there are people working on traits/interfaces - but these are still peripheral projects and not part of the core mission to my knowledge

What exactly do you mean by "traits" or "interfaces"? Why do you think these "traits" would help with the issues that bug you?

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#63

Post by ekjhgkejhgk »

jacobolus wrote: post #36
mgkuhn wrote: post #27

Julia is a very powerful and flexible language. With very powerful tools you can get a lot done quickly, including shooting yourself into the foot. Julia's type-system allows you to easily compose different elements of Julia's vast package ecosystem in ways that possibly were never tested or even intended or foreseen by the authors of these packages to be used that way. If you don't do that, you may have a much bette…

One of the basic marketing claims of the language developers is that one author's algorithm can be composed with another author's custom data type. If that's not really true in general, even for some of the most popular libraries and data types, maybe the claims should be moderated a bit.

My experience is that it is true, if you throughly implement the interfaces that your types are supposed to respect. If you dont, well, thats not the languages fault.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#64

Post by maxbond »

nsajko wrote: post #61
maxbond wrote: post #13

I'm reminded of the recent post about R's (CRAN's) somewhat radical approach to integration testing [1] and I wonder if something like that would help with the composition issues described here. [1] https://news.ycombinator.com/item?id=45259623

The Julia world is already quite careful with testing and CI. Apart from the usual unit testing, many packages do employ integration testing. The Julia project itself (compiler, etc) is tested against the package ecosystem quite often (regularly and for select pull requests).

I certainly didn't mean to imply that Julia's community was incompetent or that they were not doing integration testing. CRAN's approach (which is mandatory integration testing against all known dependents enforced by the packaging authority - the global and mandatory nature being what makes it different) is genuinely innovative and radical. I don't think that's an approach that should be adopted lightly or by most ecosystems, but I do observe that a.) these languages have similar goals and b.) it's an approach intended to solve problems of much the same shape as described in the article.

Again I think this approach is too radical for most ecosystems, but Julia is pursuing a similarly radical level of composability/reusability and evidently encountering difficulties with it, so I think there may be a compatibility there.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#65

Post by csvance »

sgaure wrote: post #59

I use julia intensively and have done so for 5 years or so. I have never encountered anything I would call a correctness bug. I guess it depends on what you count as a correctness bug, and what you mean by "julia". The core language has no obvious bugs, but there are packages of dubious quality. Say you use some package for numerical integration. One day you cook up your own floating point type, and use the same pack…

I've found that I'm most productive in Julia when minimizing the number of third party dependencies for this reason, even more so than other languages. That's not to say there are not many high quality packages available but rather the benefits of the type system align better when I have a strong understanding or control over the most pertinent interfaces. As a language Julia definitely rewards you heavily for this type of thing. Coming from Python my first instinct was to try to solve as many problems as possible with third party packages and filling in between the lines. Unsurprisingly this was the worst of both worlds.

If there was one thing I could change about Julia it most certainly wouldn't be correctness issues in my own experience. Filling in the ecosystem in terms of boring glue type stuff like a production grade gRPC client would be amazing. This was the type of problem that almost got me to give up on the language.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#66

Post by StefanKarpinski »

nsajko wrote: post #60

Julia is not without warts, but this blog post is kinda rubbish. The post claims vague but scary "correctness issues", trying to support this with a collection of unrelated issue tickets from all across Julia and the Julia package ecosystem. Not all of which were even bugs in the first place, and many of which have long been resolved. The fact that bugs happen in software should not surprise anyone. Even software of…

Number isn’t an interface—there are no operations common to all numbers. Subtyping Number is a way to opt into numeric promotion and a few other useful generic behaviors. That’s it. The fact that some abstract types are interfaces with expected behaviors, while others are dispatch points to opt into behaviors is a double edged sword: powerful and flexible, but only explicitly expressed/explained in documentation.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#67

Post by StefanKarpinski »

CyberDildonics wrote: post #48
mgkuhn wrote: post #32

Earlier quoted context omitted.

The pre-compilation speed/caching performance ("time to first plot") has practically been solved since 2024, when Julia 1.10 became the current LTS version. The current focus is on improving the generation of reasonably-sized stand-alone binaries.

I heard it for 10 years, I gave it too many chances. Each time it was solved, then it was going to be solved in a new release right around the corner, again and again. Maybe it is now, I don't care anymore.

Sad to hear that, CyberDildonics

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#68

Post by hatmatrix »

nsajko wrote: post #62
hatmatrix wrote: post #41

Earlier quoted context omitted.

I say this as a huge Julia fan, but the point is not the specific bugs in the article, but the culture of not prioritizing correctness in computation. The initial response by many (not all) in the community was look, those specific bugs are fixed; all languages have bugs; more importantly - look at the benchmark speeds of these computations! Which only reinforced this negative perception. My understanding is that it'…

> the culture of not prioritizing correctness in computation On the contrary, it is my impression the experienced Julia programmers, including those involved in JuliaLang/julia, take correctness seriously. More so than in many other PL communities. > there are people working on traits/interfaces - but these are still peripheral projects and not part of the core mission to my knowledge What exactly do you mean by "tra…

True, actually they are good in following up on numerical correctness, so I should be rephrase 'correctness in computation' to 'correctness in composition' - the types of bugs that arise from mashing a lot of modules together. On the one hand it's not a Julia issue but a package ecosystem issue.

I think you're actually even more active in the Julia community so maybe I don't have to summarize the debate but these are the types of traits and interface packages being developed that are meant to formalize how modules can be used and extended by others.

https://github.com/rafaqz/Interfaces.jl

https://discourse.julialang.org/t/interfaces-traits-in-julia...

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#69

Post by hatmatrix »

munificent wrote: post #49
hatmatrix wrote: post #41

Earlier quoted context omitted.

I say this as a huge Julia fan, but the point is not the specific bugs in the article, but the culture of not prioritizing correctness in computation. The initial response by many (not all) in the community was look, those specific bugs are fixed; all languages have bugs; more importantly - look at the benchmark speeds of these computations! Which only reinforced this negative perception. My understanding is that it'…

> the culture of not prioritizing correctness in computation. In a language with pervasive use of generic methods, I don't know what actually means. If I write a function like: function add3(x, y, z) x + y + z end Is it correct or not? What does "correct" even mean here? If you call it with values where `+` is defined on them and does what you expect, then my function probably does what you expect too. But if you pas…

I should have been more precise in my language - it's not numerical correctness but composability correctness. They won't appear in a simple example like the one you provided, but more complicated ones are provided in the original post - in the example it partially centers around how getindex should be used with a particular struct and so on.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#70

Post by hatmatrix »

patagurbon wrote: post #55
bobbylarrybobby wrote: post #37

Earlier quoted context omitted.

One can hardly call using the canonical autograd library and getting incorrect gradients or using arrays whose indices aren't 1:len and getting OOB errors “shooting oneself in the foot” — these things are supposed to Just Work, but they don't. Interfaces would go a long way towards codifying interoperability expectations (although wouldn't help with plain old correctness bugs). With regard to power and flexibility, h…

Zygote hasn’t been the “canonical” autodiff library for some time now, the community recognized its problems long ago. Enzyme and Mooncake are the major efforts and both have a serious focus on correctness

In this debate there seems to be a pervasive impulse to point out "that specific problem doesn't exist anymore," and to their credit the developers are generally good at responding to serious problems.

However, the spirit of the original post was about the lack of safeguards and cohesive direction by the community to find ways to preempt such errors. It's not an easy problem to solve since Julia's composability and flexibility adds complexity not encountered in other languages. The current solution is, 'users beware', while there are a few people working on ways to enforce correct composability. I think it's best to acknowledge that this is an ongoing issue and that it's not a problem anymore because the specific ones pointed out are fixed.

Post reply on HN