Correctness and composability bugs in the Julia ecosystem • Hacker News Board

Re: Correctness and composability bugs in the Julia ecosystem

#251

Post by DNF2 » Mon, May 16, 2022, 7:57 PM UTC

nohat wrote: post #55

The power of allowing everyone to make foundational types and functions that work together is indeed dangerous. I'm not sure you are better off in the even more dangerous waters of c/c++/fortran, except that they are older and more established with many times the man-hours sunk into them. Is there a good way to control the interaction of these many different libraries with losing the generality and composability of J…

But it's not a matter of language design. The 'choose your own indexing' is something you do entirely in libraries.

You can create your own indexing in python too, it will just be slow. The 'sin' of Julia is that it will be fast...

Re: Correctness and composability bugs in the Julia ecosystem

#252

Post by sirwhinesalot » Mon, May 16, 2022, 7:57 PM UTC

StefanKarpinski wrote: post #240

blindseer wrote: post #239

Earlier quoted context omitted.

> 2) you can find out what methods you need to implement just by running the code that uses the implementation and see what fails. For large codebases this is SO painful to do. I just don't understand how anyone gets anything done when this is how they have to develop code.

That's why interfaces are useful—they save you from that. But they don't actually solve the problem of checking that an abstraction has been implemented correctly, just that you've implemented the entire API surface area, possibly incorrectly. Note, however, that if you have a way of automatically testing the behavioral correctness of an implementation, then those tests presumably cover the entire API, so automatic t…

Indeed, static types don't save you from this issue, at least structural ones don't, exactly the same issue would occur as in Julia (see: C++ templates). Static structural types have the same problem as Julia here, you gain a lot of compositional power at the expense of potential correctness issues.

However, nominal types do "solve" the problem somewhat, as there's a clear statement of intent when you do "X implements Y" that the compiler enforces. If that promise is missing, the compiler will not let you use an X where a Y is expected. And if you do say X implements Y, then you probably tested that you did it correctly.

But this would also fail at the OffsetArray problem. The only way I can see of protecting against it (statically or dynamically) is to have an "offset-index" type, different from an integer, that you need to have to index an OffsetArray. That makes a[x] not be compatible between regular Arrays and OffsetArrays.

I don't think anyone wants that mess though. So if your language has OffsetArrays, and they're supposed to be compatible with Arrays, and you can index both the same way, no amount of static types will help (save for dependent/refinement types but those are their own mess).

EDIT: I seem to have replied to the wrong comment, but the right person, so hey, no issue in the end :)

Re: Correctness and composability bugs in the Julia ecosystem

#253

Post by grumpyprole » Mon, May 16, 2022, 7:57 PM UTC

pron wrote: post #180

grumpyprole wrote: post #174

Earlier quoted context omitted.

> Everything has correctness issues somewhere. Yes but Julia is (yet another) dynamic language, presumably for "ease of use". A language with static types would have made it easier to build correct software (scientific code in e.g. OCaml and F# can look pretty good). Julia chose a path to maximize adoption at the expense of building a reliable ecosystem. Not all languages choose to make this trade-off.

> A language with static types would have made it easier to build correct software This claim is repeated often, but numerous attempts have failed to demonstrate that this is generally the case in practice (there have been a couple of studies showing an effect in very specific circumstances). Static types might indeed assist with correctness, but they are not the only thing that does, and in some situations they coul…

> Given empirical studies, the current working hypothesis should be that if static types do have a positive effect on correctness, it is a small one.

Which use cases, languages and static type systems are you referring to? The context is very important, especially when seeking to draw general conclusions from empirical studies.

As someone who has previously posted extolling the merits of static analysis, I'm very surprised at your position regrding static types. Static types help to constrain a language and enable reasoning, either by additional static analysis or otherwise.

It is precisely the flexibility of dynamic languages that makes them difficult to reason about and difficult to build correct software in. This is why the use of dynamic languages are mostly banned in the defense industry.

Static types clearly help with composition (one of the complaints with Julia), especially at scale. How many academic empirical studies considered multimillion-line code bases? I submit for evidence a lot of expensive type-retrofitting projects such as Facebook Hack, Microsoft Typescript or Python types, which demonstrate that many companies have or had real problems with dynamic languages at any kind of scale.

Re: Correctness and composability bugs in the Julia ecosystem

#254

Post by Rayhem » Mon, May 16, 2022, 7:59 PM UTC

IshKebab wrote: post #244

ModernMech wrote: post #229

Earlier quoted context omitted.

The reason why this 0 vs 1 based indexing debate is never resolved is because all of the arguments are subjective. You've claimed definitively that "humans got it wrong", but to back up this argument you've pointed to vague notions of "elegance" and "understandability". Even Dijkstra in his argument relies on a notion of "ugliness". All such arguments fall squarely in the realm of "preferences". Just think about what…

It's not subjective. The fundamental meaning of an index is "how far are we from the start of an array". The first element is 0 from the start of the array. If humans had got this right then we wouldn't be even having this discussion. Same for similar mistakes like Pi vs Tau, negative electrons. But the mistake is understandable given that we didn't even think of 0 for a long time.

Yes, precisely. Indexing and counting are different operations.

Re: Correctness and composability bugs in the Julia ecosystem

#255

Post by a1369209993 » Mon, May 16, 2022, 7:59 PM UTC

rcthompson wrote: post #178

ChrisRackauckas wrote: post #44

Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from tho…

Is "for i in 1:length(A)" ever correct? Should Julia just emit a warning any time it encounters that pattern? Or maybe something slightly more complicated, such as that pattern followed by usage of i to index into A inside the loop?

> Is "for i in 1:length(A)" ever correct?

Yes, actually. While I have approximately zero knownledge of Julia specifically, a language-independent example might be:

  B = OneBasedArray(length(A))
  A_ = iter(A)
  for i in 1:length(A) { B[i] = pop(A_) }
  assert(iter_isdone(A_))

And if that looks contrived... yes; it is contrived.

> that pattern followed by usage of i to index into A inside the loop?

I can't think of any legitimate uses for that, but there probably are some; make sure to allow:

  len = length(A)
  for i in 1:len ...

as a `if( (x = foo()) )`-style workaround.

Re: Correctness and composability bugs in the Julia ecosystem

#256

Post by nalimilan » Mon, May 16, 2022, 8:00 PM UTC

nightpool wrote: post #103

ChrisRackauckas wrote: post #44

Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from tho…

I'll be honest, based on my experience with Julia, this makes me more worried about using e.g. libuv in production systems now, not less. I understand your opinion that "The easier it is to look at the code, the easier it is to find issues with it", but I don't think that has anything to do with the fact that `prod((Int8(100), Int8(100)))` and `prod([Int8(100), Int8(100)])` disagree, because someone decided to specia…

The problem in this case (as with most issues regarding `@inbounds`) is that this text was written before arrays with non-standard indices existed in Julia. So the example was correct at the time it was written, just like the StatsBase code was correct. Old code needs careful checking to fix all these occurrences.

Re: Correctness and composability bugs in the Julia ecosystem

#257

Post by jpeloquin » Mon, May 16, 2022, 8:06 PM UTC

SemanticStrengh wrote: post #77

one-more-minute wrote: post #62

It might be useful to separate the issues that are "just" bugs from the problems that come with Julia's unusual level of composability. I have no idea if Julia has more bog-standard, local bugs – things like data structure problems or compiler faults – than other languages of comparable maturity and resources, but clearly the OP has bumped into several, which is frustrating. The composition bugs – as in offsetarrays…

any tutorial/blog on what make julia compositionability special vs othe languages? Is there a relation with multiple dispatch or delegation?

The "Unreasonable Effectiveness of Multiple Dispatch" talk is a good example of how multiple dispatch is special in a good way, in that everything (should) work together as new types and functions are added to the ecosystem. However, this also means the scope of potential integration bugs encompasses the entire ecosystem. The Julia manual has a small section about special composibility pitfalls arising from multiple dispatch: https://docs.julialang.org/en/v1/manual/methods/#man-method-...

As best as I can summarize it: Multiple dispatch is supposed to dispatch a function call to the implementation with the most "specific" call signature. This means that you must design your functions with an eye to what everyone else has implemented or might implement so whatever function gets called does the "right" thing, and also that your implementation doesn't block someone else from writing their own implementation specialized to other types. This requires some coordination across packages, as shown in one of the manual's examples.

The rules defining type specificity (subtyping) are complicated, and I think not in the manual. They have been inferred by observation: http://janvitek.org/pubs/oopsla18a.pdf. To quote from that paper, "In many systems answering the question whether t1 <: t2 is an easy part of the development. It was certainly not our expectation, approaching Julia, that reverse engineering and formalizing the subtype relation would prove to be the challenge on which we would spend our time and energy. As we kept uncovering layers of complexity, the question whether all of this was warranted kept us looking for ways to simplify the subtype relation. We did not find any major feature that could be dropped." Julia's multiple dispatch allows a high degree of composibility, but this does create new complexity and new problems.

Re: Correctness and composability bugs in the Julia ecosystem

#258

Post by derbOac » Mon, May 16, 2022, 8:08 PM UTC

p33p wrote: post #18

Viral frequents HN so I will be curious to see if he engages this directly in a productive manor. There are many great qualities of Julia, and I've wanted to love it and use it in production. However, coming from the tooling and correctness of Rust leaves me thinking something is just missing in Julia. One of the links in the post references "cowboy" culture. While I don't think this is the correct nomenclature, ther…

I've been a user of Julia for some time (at least since beta versions). I love the language and feel like the author of the blog post is maybe exaggerating or generalizing a bit too much. On the other hand, based on my personal experiences with Julia, I can definitely empathize and feel like there's a lot about the blog post that rings true.

I share your sense that "something is just missing in Julia" but I maybe disagree with the author in that I see it as potentially changeable or something, as not hopeless.

Julia has grown tremendously in a short period of time, both in the language, its implementation, and the size of the community. So in that sense I see it as inevitable there's going to be a lot of bugs and chaos for a bit.

On the other hand, I've always felt a bit of unease that a numerical language was being developed from the ground up as that, without it being an offshoot of more general purpose language. It's not that I think there's something inherently wrong with it, but I do think that having a greater variety of perspectives looking at it are more likely to catch things early.

I don't think in this regard it's a function of academia -- although it certainly could be -- it's more a function of having a very narrow community looking at the language. Regardless of how smart they all are, I think having a broader range of perspectives might catch things earlier.

In this regard, I might have preferred the Julia fervor and effort be put into some numerical Nim libraries, or a numerical "abstracted subset of Rust" or something. It's not so much I dislike Julia as much as it is I'd feel safer with a more generalist perspective on basic language design.

But who knows. To me it's a bit ironic the author focuses on Python as an alternative, because it's not like that is free from problems, and Python has been around for a lot longer. They might be different problems, but they're not absent. Python is a bit ironic too in that it has been sort of kludged together over time into what it is today, for better or worse. I guess it feels like to me all the major numerical programming platforms have this kind of kludgy feeling in different ways; Julia feels/felt a bit like an opportunity for a clean break, if nothing else.

Re: Correctness and composability bugs in the Julia ecosystem

#259

Post by jolux » Mon, May 16, 2022, 8:09 PM UTC

pron wrote: post #180

grumpyprole wrote: post #174

Earlier quoted context omitted.

> Everything has correctness issues somewhere. Yes but Julia is (yet another) dynamic language, presumably for "ease of use". A language with static types would have made it easier to build correct software (scientific code in e.g. OCaml and F# can look pretty good). Julia chose a path to maximize adoption at the expense of building a reliable ecosystem. Not all languages choose to make this trade-off.

> A language with static types would have made it easier to build correct software This claim is repeated often, but numerous attempts have failed to demonstrate that this is generally the case in practice (there have been a couple of studies showing an effect in very specific circumstances). Static types might indeed assist with correctness, but they are not the only thing that does, and in some situations they coul…

> Note that Matlab, the workhorse of scientific computing for a few decades now, is even less typed than Julia.

You always make this argument when discussing PL features and I find it irksome. People get along fine without this feature, therefore there’s no sense in implementing it. But it cuts the other way, or we’d all still be using assembly. How many Matlab users know things could be better? Was the superiority of structured programming and avoiding GOTO ever empirically proven, or did we all just collectively realize it was a good idea?

Re: Correctness and composability bugs in the Julia ecosystem

#260

Post by patrec » Mon, May 16, 2022, 8:12 PM UTC

I tend to be a a bit wary of dynamic languages with sophisticated, performant implementations of complex abstractions, especially if they have somewhat niche appeal. In my experience this is a combination that makes for running into a lot of implementation bugs. For example, I've run into many more nasty compiler bugs with lisps (and julia at least qualifies as an almost-lisp) than with more simple-minded dynamic languages like python or erlang[1] or fairly sophisticated but niche statically typed languages.

I think watching Julia over the next few years will be quite interesting: it's the only dynamically typed language that has both sophisticated abstractions and a sophisticated implementation[1] that has enough pull to have a chance to become entrenched in certain domains. I wonder to what extent they will be able to get this problem under control.

[1] BEAM, unlike cpython, is actually a marvel of engineering and making very deliberate trade-offs. But it's not very complex.

[2] Javascript is of course the one pervasive dynamically typed programming language that has sophisticated implementations, but of mostly ill-conceived constructs.