Correctness and composability bugs in the Julia ecosystem (2022) • Hacker News Board

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#51

Post by krull10 » Tue, Sep 30, 2025, 6:09 PM UTC

hatmatrix wrote: post #43

postflopclarity wrote: post #38

Earlier quoted context omitted.

improving, still not perfect. it was true then, and is even more true now, that a large fraction of "correctness bugs" (maybe even the majority) arise from `OffsetArrays.jl`, so a simple solution besides "avoid Julia" is "avoid that package"

The greater issue that there is still no way to prevent those types of composability bugs.

The simplest approach is to always read the interface of packages one wants to use, and if one isn't provided look at the code / open an issue to interact with the developers about their input assumptions. One should also make tests to ensure the interface behaves in the expected manner when working with your code.

Using this approach since 2017 I've never really encountered the types of issues mentioned in Yuri's blog post. The biggest issue I've had is if some user-package makes a change that is effectively breaking but they don't flag it the associated release as breaking. But this isn't really a Julia issue so much as a user-space issue, and can happen in any language when relying on others' libraries.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#52

Post by krull10 » Tue, Sep 30, 2025, 6:15 PM UTC

leephillips wrote: post #47

Grikbdl wrote: post #42

Earlier quoted context omitted.

When I look, an issue such as "the base `sum!` is wrong" is still an open issue. Which, I think, is a bit ridiculous.

This is a typical example: • The documentation (currently) of the function warns not to use it this way; • This is a rather perverse use of the function(s) that would be unlikely unless you’re trying to break things; • The discussion on the issue page demonstrates the exact opposite of a culture not caring about correctness; • This kind of stuff doesn’t matter to all the scientists who are actually using Julia to do…

I've nearly exclusively used Julia since 2017. I don't think this is a perverse use of such functions -- long ago I naturally guessed I could use `cumsum!` on the same input and output and it would correctly overwrite the values (which now gives a similar warning in the documentation). However, when I first used it that way I tested if it did what I expected to verify my assumption.

It is good the documentation is now explicit that the behavior is not guaranteed in this case, but even better would be if aliasing were detected and handled (at least for base Julia arrays, so that the warning would only be needed for non-base types).

Still, the lesson is that when using generic functions one should look at what they expect of their input, and if this isn't documented one should at least test what they are giving thoroughly and not assume it just works. I've always worked this way, and never run into surprises like the types of issues reported in the blog post.

Currently there is no documentation on what properties an input to `sum!` must support in the doc string, so one needs to test its correctness when using it outside of base Julia data types (I haven't checked the broader docs for an interface specification, but if there is one it really should be linked in the docstring).

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#53

Post by leephillips » Tue, Sep 30, 2025, 6:37 PM UTC

krull10 wrote: post #52

leephillips wrote: post #47

Earlier quoted context omitted.

This is a typical example: • The documentation (currently) of the function warns not to use it this way; • This is a rather perverse use of the function(s) that would be unlikely unless you’re trying to break things; • The discussion on the issue page demonstrates the exact opposite of a culture not caring about correctness; • This kind of stuff doesn’t matter to all the scientists who are actually using Julia to do…

I've nearly exclusively used Julia since 2017. I don't think this is a perverse use of such functions -- long ago I naturally guessed I could use `cumsum!` on the same input and output and it would correctly overwrite the values (which now gives a similar warning in the documentation). However, when I first used it that way I tested if it did what I expected to verify my assumption. It is good the documentation is no…

But your use of cumsum!() seems natural; I can see using it that way, and might have done so myself. The use of sum!() under discussion seems weird, though.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#54

Post by missinglugnut » Tue, Sep 30, 2025, 6:51 PM UTC

postflopclarity wrote: post #38

ForceBru wrote: post #35

Has anything changed since then? What are y'all's thoughts about correctness in Julia in 2025?

improving, still not perfect. it was true then, and is even more true now, that a large fraction of "correctness bugs" (maybe even the majority) arise from `OffsetArrays.jl`, so a simple solution besides "avoid Julia" is "avoid that package"

It baffles me that they dug this hole in the first place. I have feelings on the zero-indexing vs one-indexing debate, but at the end of the day you can write correct code in either, as long as you know which one you're using.

But Julia fucked it up to where it's not clear what you're using, and library writers don't know which one has been passed! It's insane. They chose style over consistency and correctness and it's caused years of suffering.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#55

Post by patagurbon » Tue, Sep 30, 2025, 6:51 PM UTC

bobbylarrybobby wrote: post #37

mgkuhn wrote: post #27

Julia is a very powerful and flexible language. With very powerful tools you can get a lot done quickly, including shooting yourself into the foot. Julia's type-system allows you to easily compose different elements of Julia's vast package ecosystem in ways that possibly were never tested or even intended or foreseen by the authors of these packages to be used that way. If you don't do that, you may have a much bette…

One can hardly call using the canonical autograd library and getting incorrect gradients or using arrays whose indices aren't 1:len and getting OOB errors “shooting oneself in the foot” — these things are supposed to Just Work, but they don't. Interfaces would go a long way towards codifying interoperability expectations (although wouldn't help with plain old correctness bugs). With regard to power and flexibility, h…

Zygote hasn’t been the “canonical” autodiff library for some time now, the community recognized its problems long ago. Enzyme and Mooncake are the major efforts and both have a serious focus on correctness

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#56

Post by postflopclarity » Tue, Sep 30, 2025, 7:11 PM UTC

Grikbdl wrote: post #46

TimorousBestie wrote: post #29

Earlier quoted context omitted.

Python also suffers from such problems. For example, here’s scipy’s issue tracker, filtered for bugs only: https://github.com/scipy/scipy/issues?q=is%3Aissue%20state%3... Scrolling through this list, it’s clear that many are “correctness issues.” I do not link this to argue that scipy bugs are more serious or more frequent. I don’t think that kind of statistical comparison is meaningful. However, I think a motivated…

Explicit errors are not correctness issues. Some numerical instability issues in certain algorithms in certain corner cases might be considered as such though. Regardless, overall, these are grossly of another complexity and seriousness than the base sum function being just wrong, or cultural issues among developers with not verifying inputs "for performance", or things of that nature. The scientific Python community…

I have found multiple "correctness bugs" of equal seriousness in Polars, and one of them is still open. that is not to throw shade at polars --- I love that package! but my point is that these things happen everywhere in software.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#57

Post by yurivish » Tue, Sep 30, 2025, 7:36 PM UTC

I'm the author of the original post. I still like and miss my friends in the Julia community, but the technical and cultural issues persist.

I may refresh the post with more recent information at some point. In the meantime, those curious can find a short story of one newer correctness bug here: https://discourse.julialang.org/t/why-is-it-reliable-to-use-...

The person who eventually fixed the issue, mkitti, had to push through a lot of "institutional" friction to do so, and the eventual fix is the result of his determined efforts.

While his part of the story mostly played out in venues outside of the Discourse forum some of it is on display in this thread: https://discourse.julialang.org/t/csv-jl-findmax-and-argmax-...

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#58

Post by csvance » Tue, Sep 30, 2025, 7:56 PM UTC

missinglugnut wrote: post #54

postflopclarity wrote: post #38

Earlier quoted context omitted.

improving, still not perfect. it was true then, and is even more true now, that a large fraction of "correctness bugs" (maybe even the majority) arise from `OffsetArrays.jl`, so a simple solution besides "avoid Julia" is "avoid that package"

It baffles me that they dug this hole in the first place. I have feelings on the zero-indexing vs one-indexing debate, but at the end of the day you can write correct code in either, as long as you know which one you're using. But Julia fucked it up to where it's not clear what you're using, and library writers don't know which one has been passed! It's insane. They chose style over consistency and correctness and it…

Technically you don't need to know what array indexing is being used if you iterate using firstindex(arr):lastindex(arr). AFAIK the issue was that this wasn't consistently done across the Julia ecosystem including parts of the standard library at the time. No clue as to whether this still holds true, but I don't worry about it because I don't use OffsetArrays.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#59

Post by sgaure » Tue, Sep 30, 2025, 8:21 PM UTC

I use julia intensively and have done so for 5 years or so. I have never encountered anything I would call a correctness bug. I guess it depends on what you count as a correctness bug, and what you mean by "julia". The core language has no obvious bugs, but there are packages of dubious quality.

Say you use some package for numerical integration. One day you cook up your own floating point type, and use the same package with success. Then you change your floating point type subtly, and suddenly weird things start to happen. Is it a correctness bug? Whose bug?

Surely, the author of the integration package didn't have your weird floating type in mind, but it still worked. Until you made it even weirder. These are the things some people think are correctness bugs in julia. It's mostly poor coding.

Re: Correctness and composability bugs in the Julia ecosystem (2022)

#60

Post by nsajko » Tue, Sep 30, 2025, 8:52 PM UTC

Julia is not without warts, but this blog post is kinda rubbish. The post claims vague but scary "correctness issues", trying to support this with a collection of unrelated issue tickets from all across Julia and the Julia package ecosystem. Not all of which were even bugs in the first place, and many of which have long been resolved.

The fact that bugs happen in software should not surprise anyone. Even software of critical importance, such as GCC or LLVM, whose correctness is relied upon by the implementations of many programming languages (including C, C++ and Julia itself), are buggy.

Instead the post could have focused more on actual design issues, such as some of the Base interfaces being underspecified:

> the nature of many common implicit interfaces has not been made precise (for example, there is no agreement in the Julia community on what a number is)

The underspecified nature of Number (or Real, or IO) is an issue, albeit not related with the rest of the blog post. It does not excuse the scaremongering in the blog post, however.