Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem (2022)

yuri.is

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#71

Post by hatmatrix »

krull10 wrote: post #51
hatmatrix wrote: post #43

Earlier quoted context omitted.

The greater issue that there is still no way to prevent those types of composability bugs.

The simplest approach is to always read the interface of packages one wants to use, and if one isn't provided look at the code / open an issue to interact with the developers about their input assumptions. One should also make tests to ensure the interface behaves in the expected manner when working with your code. Using this approach since 2017 I've never really encountered the types of issues mentioned in Yuri's bl…

It doesn't happen so frequently in practice, which is maybe why it's not felt equally by everyone in the community. I've followed some discussions on this topic and the current solution is, as you say, to place the onus entirely on the user. There is another faction that wants to support the user through the IDE to warn them of errors they might be making, and others that want to implement formal specifications that catch the errors during specifications. It's not an easy problem to fix and therefore the "simplest approach" as you describe remains the leading solution.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#72

Post by nsajko »

hatmatrix wrote: post #68
nsajko wrote: post #62

Earlier quoted context omitted.

> the culture of not prioritizing correctness in computation On the contrary, it is my impression the experienced Julia programmers, including those involved in JuliaLang/julia, take correctness seriously. More so than in many other PL communities. > there are people working on traits/interfaces - but these are still peripheral projects and not part of the core mission to my knowledge What exactly do you mean by "tra…

True, actually they are good in following up on numerical correctness, so I should be rephrase 'correctness in computation' to 'correctness in composition' - the types of bugs that arise from mashing a lot of modules together. On the one hand it's not a Julia issue but a package ecosystem issue. I think you're actually even more active in the Julia community so maybe I don't have to summarize the debate but these are…

What I wanted to say is that I'm skeptical regarding "interfaces", either as a language feature or as a package. Although TBH I have not yet given any specific "interfaces" design more than a cursory glance, so my position is not really justified.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#73

Post by nsajko »

maxbond wrote: post #64
nsajko wrote: post #61

Earlier quoted context omitted.

The Julia world is already quite careful with testing and CI. Apart from the usual unit testing, many packages do employ integration testing. The Julia project itself (compiler, etc) is tested against the package ecosystem quite often (regularly and for select pull requests).

I certainly didn't mean to imply that Julia's community was incompetent or that they were not doing integration testing. CRAN's approach (which is mandatory integration testing against all known dependents enforced by the packaging authority - the global and mandatory nature being what makes it different) is genuinely innovative and radical. I don't think that's an approach that should be adopted lightly or by most e…

I don't think testing against every existing dependent would make sense currently. The issue is the lack of tooling for mechanically checking whether the dependent accesses implementation details of the dependency, in which case it would be valid for the dependency to break the dependent.

There are some proposals to forbid the registration of a package release which trespasses on the internals of another package, though.

I hope someone tackles the above sooner or later, but another issue is the approach of testing every known dependent package might be very costly, both in terms of compute and manual labor, the latter because someone would have to do the work of maintaining a blacklist for packages with flaky unit tests. The good news is that this work might considerably overlap with the already existing PkgEval infrastructure. We'll see.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#74

Post by nsajko »

StefanKarpinski wrote: post #66
nsajko wrote: post #60

Julia is not without warts, but this blog post is kinda rubbish. The post claims vague but scary "correctness issues", trying to support this with a collection of unrelated issue tickets from all across Julia and the Julia package ecosystem. Not all of which were even bugs in the first place, and many of which have long been resolved. The fact that bugs happen in software should not surprise anyone. Even software of…

Number isn’t an interface—there are no operations common to all numbers. Subtyping Number is a way to opt into numeric promotion and a few other useful generic behaviors. That’s it. The fact that some abstract types are interfaces with expected behaviors, while others are dispatch points to opt into behaviors is a double edged sword: powerful and flexible, but only explicitly expressed/explained in documentation.

> Number isn’t an interface—there are no operations common to all numbers.

When creating a new type, it should be more clear cut when is subtyping Number (or Real, etc.) valid. Should unitful quantities be numbers? Should intervals be numbers? Related: I think there are some attempts by Tim Holy and others to create/document "thick numbers".

Furthermore, I believe it might be good to align the Number type hierarchy with math/abstract algebra as much as possible without breaking backwards compatibility, which might making Number, or some subtypes of it, actual interfaces.

> Subtyping Number is a way to opt into numeric promotion and a few other useful generic behaviors. That’s it.

OK, but I think that's not documented either.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#75

Post by postflopclarity »

yurivish wrote: post #57

I'm the author of the original post. I still like and miss my friends in the Julia community, but the technical and cultural issues persist. I may refresh the post with more recent information at some point. In the meantime, those curious can find a short story of one newer correctness bug here: https://discourse.julialang.org/t/why-is-it-reliable-to-use-... The person who eventually fixed the issue, mkitti, had to p…

while I certainly agree that bug was a bit frustrating, in all that

* it happened in the first place

* it took so long to get noticed

* it took so long to merge the fix after being reported

I do feel like I should push back on the term "institutional friction." it was more of a bus factor problem; there were not enough (aka zero) maintainer eyeballs on the proposed fix. but there wasn't exactly anybody saying not to fix it, which is what I think of as friction.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#76

Post by StefanKarpinski »

nsajko wrote: post #74
StefanKarpinski wrote: post #66

Earlier quoted context omitted.

Number isn’t an interface—there are no operations common to all numbers. Subtyping Number is a way to opt into numeric promotion and a few other useful generic behaviors. That’s it. The fact that some abstract types are interfaces with expected behaviors, while others are dispatch points to opt into behaviors is a double edged sword: powerful and flexible, but only explicitly expressed/explained in documentation.

> Number isn’t an interface—there are no operations common to all numbers. When creating a new type, it should be more clear cut when is subtyping Number (or Real, etc.) valid. Should unitful quantities be numbers? Should intervals be numbers? Related: I think there are some attempts by Tim Holy and others to create/document "thick numbers". Furthermore, I believe it might be good to align the Number type hierarchy w…

Yeah, probably should be documented.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#77

Post by munificent »

hatmatrix wrote: post #69
munificent wrote: post #49

Earlier quoted context omitted.

> the culture of not prioritizing correctness in computation. In a language with pervasive use of generic methods, I don't know what actually means. If I write a function like: function add3(x, y, z) x + y + z end Is it correct or not? What does "correct" even mean here? If you call it with values where `+` is defined on them and does what you expect, then my function probably does what you expect too. But if you pas…

I should have been more precise in my language - it's not numerical correctness but composability correctness. They won't appear in a simple example like the one you provided, but more complicated ones are provided in the original post - in the example it partially centers around how getindex should be used with a particular struct and so on.

There's nothing numerical in my comment. There's just arguments and function calls, which happened to be named "+".

My point is that there is an implied contract that `add3()` only does what you expect if you pass it values where `+` happens to do what you expect. When you have a language with fully open generic methods like Julia, it's very powerful, but the trade-off is that every function is effectively like middleware where all it can really say is "if you give me things that to delegate to the right things, I'll do the right thing too".

When I'm writing `add3()`, I don't know what `+` does. I'm writing a function in terms of open-ended abstractions that I don't control, so it's very hard to make any promises about the semantics of my function.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#78

Post by yurivish »

munificent wrote: post #77
hatmatrix wrote: post #69

Earlier quoted context omitted.

I should have been more precise in my language - it's not numerical correctness but composability correctness. They won't appear in a simple example like the one you provided, but more complicated ones are provided in the original post - in the example it partially centers around how getindex should be used with a particular struct and so on.

There's nothing numerical in my comment. There's just arguments and function calls, which happened to be named "+". My point is that there is an implied contract that `add3()` only does what you expect if you pass it values where `+` happens to do what you expect. When you have a language with fully open generic methods like Julia, it's very powerful, but the trade-off is that every function is effectively like middl…

Your perspective tracks with mine. Without contracts, either specified in documentation or as static guarantees, it is hard or impossible to build robust programs.

In Julia it's almost as if every function is an interface, with (usually quite terse) documentation as its only semantic constraint. For example, here is the full documentation for `+`: https://docs.julialang.org/en/v1/base/math/#Base.:+

I love Game Programming Patterns, by the way! Laughed out loud when I first saw the back cover.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#79

Post by munificent »

yurivish wrote: post #78
munificent wrote: post #77

Earlier quoted context omitted.

There's nothing numerical in my comment. There's just arguments and function calls, which happened to be named "+". My point is that there is an implied contract that `add3()` only does what you expect if you pass it values where `+` happens to do what you expect. When you have a language with fully open generic methods like Julia, it's very powerful, but the trade-off is that every function is effectively like middl…

Your perspective tracks with mine. Without contracts, either specified in documentation or as static guarantees, it is hard or impossible to build robust programs. In Julia it's almost as if every function is an interface, with (usually quite terse) documentation as its only semantic constraint. For example, here is the full documentation for `+`: https://docs.julialang.org/en/v1/base/math/#Base.:+ I love Game Progra…

> Without contracts, either specified in documentation or as static guarantees, it is hard or impossible to build robust programs.

Right. I think a big part of this is expectation management. Julia lets you compose unrelated libraries much more freely than most other languages do. That's very powerful, but if you come into it expecting all of those compositions to magically work, I think you just have an unrealistic expectation.

There's no silver bullet when it comes to code reuse and Conway's Law can't be entirely avoided.

> I love Game Programming Patterns, by the way! Laughed out loud when I first saw the back cover.

:D

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#80

Post by yurivish »

munificent wrote: post #79
yurivish wrote: post #78

Earlier quoted context omitted.

Your perspective tracks with mine. Without contracts, either specified in documentation or as static guarantees, it is hard or impossible to build robust programs. In Julia it's almost as if every function is an interface, with (usually quite terse) documentation as its only semantic constraint. For example, here is the full documentation for `+`: https://docs.julialang.org/en/v1/base/math/#Base.:+ I love Game Progra…

> Without contracts, either specified in documentation or as static guarantees, it is hard or impossible to build robust programs. Right. I think a big part of this is expectation management. Julia lets you compose unrelated libraries much more freely than most other languages do. That's very powerful, but if you come into it expecting all of those compositions to magically work, I think you just have an unrealistic…

> Julia lets you compose unrelated libraries much more freely than most other languages do. That's very powerful, but if you come into it expecting all of those compositions to magically work, I think you just have an unrealistic expectation.

Yep, and it is unfortunate that this unrealistic expectation is explicitly encouraged by the creators of the language:

> It is actually the case in Julia that you can take generic algorithms that were written by one person and custom types that were written by other people and just use them together efficiently and effectively.

It seems worth reiterating that on a personal level I really like and appreciate the vast majority of the folks I’ve met in the Julia community. I’m glad I got to hang out with them and learn from them. But in my opinion setting expectations like this fosters bad science.

Post reply on HN