Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

41–50 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#41
Most of these seem to be about packages in the ecosystem (which, after clicking through all links, actually almost all got fixed in a very timely manner, sometimes already in a newer version of the packages than the author was using), not about the language itself. Other than that, the message of this seems to be "newer software has bugs", which yes is a thing..?

For example, the majority of issues referenced are specific to a single package, StatsBase.jl - which apparently was written before OffsetArrays.jl was a thing and thus is known to be incompatible:

> Yes, lots of JuliaStats packages have been written before offset axes existed. Feel free to make a PR adding checks.

https://github.com/JuliaStats/StatsBase.jl/issues/646#issuec...

EDIT: Since this comment seems to gain some traction - title is editorialized, original is "Why I no longer recommend Julia".

Re: Correctness and composability bugs in the Julia ecosystem

#42

I mean this looks like good potential targets to improve the language moving forward, it's healthy to not be in awe of your tools and push to make them better. I don't see this as "bad" honestly.

I think the real test will be whether or not Julia's custodians / developers start putting a greater focus on semantics and correctness. When a language's raison d'être is to try out certain ideas, it probably makes sense for a while to ignore corner cases and rigor. But as the author points out, they eventually become gating factors for wider adoption.

It's still at version 1.x, maybe an explicit roadmap could help tackling those issues?

Re: Correctness and composability bugs in the Julia ecosystem

#43
post #19

Quoted post unavailable.

Genuinely curious (I don’t work with low level systems day-to-day), but why would C/C++ be better for safety?

I would imagine that the risk of bugs related to memory access and data racing would be pretty high, compared to a higher-level language or something like Rust that focuses on safety.

Re: Correctness and composability bugs in the Julia ecosystem

#44
Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from those three points, that clearly points out tons of cases where Python, R, etc. code is all incorrect where Julia isn't.

I think what's interesting about Julia is that because the code is all Julia, it's really easy to dig in there and find potential bugs. The standard library functions can be accessed with @edit sum(1:5) and there you go, hack away. The easier it is to look at the code, the easier it is to find issues with it. This is why Julia has such a higher developer to user ratio. That has its pros and cons of course. It democratizes the development process, but it means that people who don't have a ton of development experience (plus Fortran or C knowledge) are not excluded from contributing. Is that good or bad? Personally I believe it's good in the long run, but can have its bumps.

As an aside, the author highlights "for i in 1:length(A)". I agree, code should never do that. It should be `eachindex(A)`. In general things should use iterators which are designed for arbitrary indexing based on iterators. This is true in any language, though you'll always have some newcomers write code (and documentation) with this. Even experienced people who don't tend to use arrays beyond Array tend to do this. It's an interesting issue because coding style issues perpetuate themselves: explicitly using 1-base wasn't an issue before GPUs and OffsetArrays, but then loop code like that trains the next generation, and so more people use it. In the end the people who really know to handle these cases are the people who tend to use these cases, just like how people who write in styles that are ARM-safe tend to be people who use ARM. Someone should just run a bot that opens a PR for every occurrence of this (especially in Base), as that would then change the source that everyone learns from and completely flip the style.

Re: Correctness and composability bugs in the Julia ecosystem

#45
post #40

Earlier quoted context omitted.

I was wondering if the 1-based arrays (and option to change index base) would factor into this. > OffsetArrays in particular proved to be a strong source of correctness bugs. The package provides an array type that leverages Julia’s flexible custom indices feature to create arrays whose indices don’t have to start at zero or one. Array indexing is such a core thing and I don't understand why anything mathematical or…

> Array indexing is such a core thing and I don't understand why anything mathematical or scientific would start with 1. Because starting with 0 is neither math nor array indexing in general. It's just how the base addresss of an array pointer memory block was referenced in C (and it spread from there). Which is why all math focused languages use 1-based (fortran, apl, matlab, r, mathematica, etc.)

> C (and it spread from there).

BCPL.

Re: Correctness and composability bugs in the Julia ecosystem

#46
Not specific to specific examples in the article, I think some of the things people perceive as "bugs" other people see as features or an opportunity to correct past mistakes.

I can remember an example where I suggested automatic treatment of missing values in a stats library, and the library maintainer disagreed. Meaning, my lobbying for Julia to do what R/Python did was seen as "Yes, but that's wrong and we shouldn't promote that sort of treatment". As a business user, I didn't care that it was theoretically wrong, the maintainer as an academic did.

That ends up becoming open-source prerogative. I could do it wrong "on my own time" in my own code...doesn't make either a bug, but a different choice based on perspective.

Re: Correctness and composability bugs in the Julia ecosystem

#48
So it seems Julia's multiple dispatch (dynamic dispatch for any function based on argument types) has a flaw: namely, if the types used do not match assumptions present in the implementation of the function (e.g. arrays start at 1), the results may be silently incorrect. Julia's multiple dispatch is really cool, but I'm not sure how this issue can be prevented in practice (without a lot of added verbosity). It'd be a pity to have to restrict yourself to a small set of types you know work with the functions you're using, because multiple dispatch is one of Julia's killer features.

Re: Correctness and composability bugs in the Julia ecosystem

#49
post #4

Wait, are those examples real? I remember complaining about 1-bsaed indexing only to be told "julia is great! we have offsetindex". If it's a source of bugs, that ... greatly reduces my future interest in adopting the language.

The problem isn't that 1-base indexing can be "fixed" in Julia. The problem is that you see 1-based indexing as a flaw.

I didn't say I see 1-based indexing as a flaw. I said I complained about it, and then learned they supported multiple types of offsets (which ostensibly resolved the issue for me), only to learn that the stats library was "written before offsetindex" and still has bugs related to it.

Re: Correctness and composability bugs in the Julia ecosystem

#50
post #24

Oftentimes people describe languages as "Turing complete" but how often do they talk about languages being "Gödel incomplete?" Another way of stating maybe is "Are what some call flaws what others call features?" https://stackoverflow.com/questions/7284/what-is-turing-comp... https://plato.stanford.edu/entries/goedel-incompleteness/

Even fairly simple arithmetic is incomplete, so unless the language is heavily restricted, allowing only multiplication of positive integers (x)or addition of natural numbers, they're all going to be incomplete.
Post reply on HN