Correctness and composability bugs in the Julia ecosystem • Hacker News Board

Re: Correctness and composability bugs in the Julia ecosystem

#241

Post by cozzyd » Mon, May 16, 2022, 7:23 PM UTC

snicker7 wrote: post #175

cozzyd wrote: post #163

Earlier quoted context omitted.

In my experience it's much harder to write Python that won't crash than C/C++ that won't crash. With Python, rarely taken code paths can have very dumb errors in them (e.g. accidentally introducing a new variable with a different name than the variable you wanted or typoing a method name, or passing arguments with the wrong type) that would get caught by a C/C++ compiler but won't be caught in Python until the crash…

C/C++ memory errors are literally responsible for 70% of ALL critical security bugs. Most python "dumb errors" can be caught by linters. Memory is infinitely harder. Source -- https://msrc-blog.microsoft.com/2019/07/16/a-proactive-appro...

The linter can't tell that the method doesn't exist until runtime, can it? If so, it wouldn't accept lots of valid python... Also, what linter should I use?

Either way, not all software needs to be secure to an adversary (since it's running behind several firewalls and if the attacker has shell access, then it's already game over).

Re: Correctness and composability bugs in the Julia ecosystem

#242

Post by tialaramex » Mon, May 16, 2022, 7:27 PM UTC

Sukera wrote: post #226

StefanKarpinski wrote: post #222

Earlier quoted context omitted.

The big language design problem that I think this post highlights is that the flip side of Julia's composability is that composing generic code with types that implement abstractions can easily expose bugs when the caller and the callee don't agree on exactly what the abstraction is. Several of the bugs that Yuri reported are a very specific case of this: there's a lot of generic code that assumes that array indexing…

To expand on the "interfaces are not enough" part: Defining an interface on an abstract type only gives you that a implementation exists, not that it is correct , i.e. that the specific implementation for a subtype guarantees the same properties the interface specifies. On top of this, you really want to be alerted to when you expect more of an interface than the interface guarantees - this is what happened in the ca…

> Defining an interface on an abstract type only gives you that a implementation exists, not that it is correct

Pretty far off topic for Julia, but the definition of Rust's Traits over semantics rather than syntax (even though of course the compiler will only really check your syntax) gives me a lot of this.

The fact that this Bunch claims to be IntoIterator tells me that the person who implemented that explicitly intends that I can iterate over the Doodads. They can't accidentally be IntoIterator the author has to literally write the implementation naming the Trait to be implemented.

But that comes at a heavy price of course, if the author of Bunch never expected me to iterate over it, the best I can do is new type MyBunch and implement IntoIterator using whatever ingredients are provided on the surface of Bunch. This raises the price of composition considerably :/

> you really want to be alerted to when you expect more of an interface than the interface guarantees

In the case alluded to (AbstractArray) I feel like the correct thing was not to implement the existing interface. That might have been disruptive at the time, but people adopting a new interface which explicitly warns them not to 1:length(A) are not likely to screw this up, and by now perhaps everything still popular would have upgraded.

Re-purposing existing interfaces is probably always a bad idea, even if you can persuade yourself it never specifically said it was OK to use it the way you suspect everybody was in practice using it, Hyrum's Law very much applies. That interface is frozen in place, make a new one.

Re: Correctness and composability bugs in the Julia ecosystem

#243

Post by ble » Mon, May 16, 2022, 7:30 PM UTC

This article contains no instances of the word "test", which seems surprising but entirely in keeping with the author's observations.

> Julia has no formal notion of interfaces, generic functions tend to leave their semantics unspecified in edge cases, and the nature of many common implicit interfaces has not been made precise (for example, there is no agreement in the Julia community on what a number is).

> The Julia community is full of capable and talented people who are generous with their time, work, and expertise. But systemic problems like this can rarely be solved from the bottom up, and my sense is that the project leadership does not agree that there is a serious correctness problem. They accept the existence of individual isolated issues, but not the pattern that those issues imply.

It sounds like the cultural standard for writing libraries is, "works good enough for users like me" which should be good if you are using things the same way as the authors. Writing good tests for numerics is hard and grueling; testing numerics or numerics-like code is not nearly as fun or productive-feeling as using numerics to get shit done, so it all makes sense to me.

Re: Correctness and composability bugs in the Julia ecosystem

#244

Post by IshKebab » Mon, May 16, 2022, 7:32 PM UTC

ModernMech wrote: post #229

IshKebab wrote: post #207

Earlier quoted context omitted.

Yeah because humans got it wrong. Really the word for "first" should correspond to the number 0. Try doing a block iteration over an array, or any kind of interval algorithms in 0-based and 1-based. 0-based with right-open intervals just results in way way more elegant, easier to understand and (very) slightly more efficient code.

The reason why this 0 vs 1 based indexing debate is never resolved is because all of the arguments are subjective. You've claimed definitively that "humans got it wrong", but to back up this argument you've pointed to vague notions of "elegance" and "understandability". Even Dijkstra in his argument relies on a notion of "ugliness". All such arguments fall squarely in the realm of "preferences". Just think about what…

It's not subjective. The fundamental meaning of an index is "how far are we from the start of an array". The first element is 0 from the start of the array.

If humans had got this right then we wouldn't be even having this discussion. Same for similar mistakes like Pi vs Tau, negative electrons. But the mistake is understandable given that we didn't even think of 0 for a long time.

Re: Correctness and composability bugs in the Julia ecosystem

#245

Post by buescher » Mon, May 16, 2022, 7:37 PM UTC

mirekrusin wrote: post #235

buescher wrote: post #203

Earlier quoted context omitted.

And Python is implemented as a large C program.

66% python, 32% C - for cpython. 94% python, 5% C - for pypy.

cpython is the reference implementation and 32% means about 350,000 lines of code. My point stands.

Re: Correctness and composability bugs in the Julia ecosystem

#246

Post by StefanKarpinski » Mon, May 16, 2022, 7:38 PM UTC

pron wrote: post #180

grumpyprole wrote: post #174

Earlier quoted context omitted.

> Everything has correctness issues somewhere. Yes but Julia is (yet another) dynamic language, presumably for "ease of use". A language with static types would have made it easier to build correct software (scientific code in e.g. OCaml and F# can look pretty good). Julia chose a path to maximize adoption at the expense of building a reliable ecosystem. Not all languages choose to make this trade-off.

> A language with static types would have made it easier to build correct software This claim is repeated often, but numerous attempts have failed to demonstrate that this is generally the case in practice (there have been a couple of studies showing an effect in very specific circumstances). Static types might indeed assist with correctness, but they are not the only thing that does, and in some situations they coul…

In particular, not a single issue mentioned in this article would have been prevented by static type checking.

Re: Correctness and composability bugs in the Julia ecosystem

#247

Post by adgjlsfhk1 » Mon, May 16, 2022, 7:40 PM UTC

patrickkidger wrote: post #231

KenoFischer wrote: post #108

So this one is a tough one for me, because Yuri has certainly spent significant time with Julia and I think he's a very competent programmer, so his criticism is certainly to be taken seriously and I'm sad to hear he ended up with a sour opinion. There's a lot of different issues mentioned in the post, so I'm not really sure what angle to best go at it from, but let me give it a shot anyway. I think there's a couple…

FWIW my take is not that Yuri is expressing "there are too many bugs" so much as he's expressing a problem in the culture surrounding Julia itself: > But systemic problems like this can rarely be solved from the bottom up, and my sense is that the project leadership does not agree that there is a serious correctness problem. Concisely: 1. The ecosystem is poorly put together. (It's been produced by academics rather t…

I don't think it needs a rewrite as much as careful maintenance from people who have time to dedicate to software quality. Most of the APIs are good, it's just that a lot of the code is under-tested and doesn't receive enough love. Having more big companies using Julia would help a lot with that.

Re: Correctness and composability bugs in the Julia ecosystem

#248

Post by jbezanson » Mon, May 16, 2022, 7:44 PM UTC

I'm not sure what to make of this. Yuri is great and I'll certainly miss having him in the Julia community. Yes, of course there are bugs. We work on fixing them all the time. If there are just too many for you, or we are too slow at fixing them for you, then OK I understand you might walk away.

With these kinds of posts (and the reactions to them) lots of issues tend to get conflated. For example there are issues with OffsetArrays because some people write code assuming indexes start at 1. Starting at 0 wouldn't fix that. A static type system wouldn't fix that; most static type systems don't check array bounds. Are we supposed to un-register the OffsetArrays package? Should we disallow overloading indexing? Personally I have told people not to use `@inbounds` many times. We could remove it, but those who want the last drop of performance would not be too happy. The only path I see is to fix the bugs.

> They accept the existence of individual isolated issues, but not the pattern that those issues imply.

I admit, I do not see the pattern allegedly formed by these issues. Of course, static types do remove a whole category of issues, but "switch to static types" is not really a practical request. There are other things you can do, like testing, but we do a LOT of testing. I really do not mean to downplay Yuri's experience here, I am just not sure what to take away other than that we should work even harder on bugs and quality.

Re: Correctness and composability bugs in the Julia ecosystem

#249

Post by lmiq » Mon, May 16, 2022, 7:51 PM UTC

runevault wrote: post #202

mattkrause wrote: post #97

Earlier quoted context omitted.

There's a parallel idea, that you should avoid--insofar as is possible--numerical indexing. In other words, instead of iterating over `0:length(X) - 1` or `1:length(X)`, you use something like `for element in array` or indices = CartesianIndices(multidimensional_X) for index in indices X[index] = # whatever If you do that, you don't need to keep track of whether it's zero-based, one-based, or anything else. In fact,…

I'm only skimming this post and I'm not familiar with Julia so maybe I'm missing it, but does it have a way to get an item AND its index? There's I think Enumrable? in Rust where it gives you a tuple with both the item and its index in cases where you need both.

for (i,val) in pairs(array)

Re: Correctness and composability bugs in the Julia ecosystem

#250

Post by ThenAsNow » Mon, May 16, 2022, 7:55 PM UTC

pron wrote: post #180

grumpyprole wrote: post #174

Earlier quoted context omitted.

> Everything has correctness issues somewhere. Yes but Julia is (yet another) dynamic language, presumably for "ease of use". A language with static types would have made it easier to build correct software (scientific code in e.g. OCaml and F# can look pretty good). Julia chose a path to maximize adoption at the expense of building a reliable ecosystem. Not all languages choose to make this trade-off.

> A language with static types would have made it easier to build correct software This claim is repeated often, but numerous attempts have failed to demonstrate that this is generally the case in practice (there have been a couple of studies showing an effect in very specific circumstances). Static types might indeed assist with correctness, but they are not the only thing that does, and in some situations they coul…

We can trade anecdotes on this topic, but I've written numerical code in OCaml and also Julia. The strictness of OCaml's type system is painful in a numerical context but for virtually all other things it is awesome to pass code into the interpreter/compiler and catch structural problems at compile-time rather than maybe at runtime.

OCaml's type system is almost certainly not the right model for Julia but the ad-hoc typing/interface system Julia currently employs is at strong odds with compile-time correctness. There's almost certainly some middle ground to be discovered which might be unsound in a strict sense but pragmatically constrains code statically so there is high likelihood of having to go out of your way to pull the footgun trigger.

You can see how little type annotations are used in practice in major Julia libraries. It should be integral to best practice in the language to specify some traits/constraints that arguments must satisfy to be semantically valid, but what you often see instead is a (potentially inscrutable) runtime error.