Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

Post reply on HN
Print view View original

291–300 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#291

Post by dan-robertson »

TimTheTinker wrote: post #185
rcthompson wrote: post #178

Earlier quoted context omitted.

Is "for i in 1:length(A)" ever correct? Should Julia just emit a warning any time it encounters that pattern? Or maybe something slightly more complicated, such as that pattern followed by usage of i to index into A inside the loop?

> Is "for i in 1:length(A)" ever correct? In some rare cases, it very well might be exactly what the code's author intended and needed. I tend to lean towards when Martin Fowler calls an "enabling attitude"[0] (as opposed to a "directing attitude") -- that is, when faced with a choice about how to design the primitives of an interface, I lean more often towards providing flexibility, and I try to avoid choosing ahead…

Why not have a feature to allow you to turn off the warning? E.g. have something recognise 1:length(x) and complain unless you write e.g. @nowarn eachindex before it.

Re: Correctness and composability bugs in the Julia ecosystem

#292

Post by FabHK »

CRConrad wrote: post #288
FabHK wrote: post #171

Earlier quoted context omitted.

Yes. A German friend of mine moved into her student dormitory in the US, and when she was told that her room was on the first floor, asked whether there was a lift, because she had a heavy suitcase... Having said that, given that there are basements (in Europe, at least), it makes sense to call the ground floor 0. We are dealing with integers here, not natural numbers.

But floors of multi-storey buildings are a pretty unique exception in the real world in having a characteristic where zero -- the number of stairs you need to climb from the ground floor -- has an actual tangible meaning (on the ground floor). How many other such examples can you (editorial you; anyone) come up with? Not many, I'd bet.

I thought you'd be wrong, and immediately came up with:

- Hours after midnight. The (non-anglosaxon) watch goes from 0:00 to 24:00 (the latter is useful for deadlines: The proposal must be submitted by Friday, 24:00 (which coincides with Saturday, 0:00)).

Then I cheated and looked up Wikipedia (https://en.wikipedia.org/wiki/Zero-based_numbering#Other_fie... ) - and whoops, that's basically it!

Re: Correctness and composability bugs in the Julia ecosystem

#293

Post by CRConrad »

asdfman123 wrote: post #183

> In my experience, Julia and its packages have the highest rate of serious correctness bugs of any programming system I’ve used, and I started programming with Visual Basic 6 in the mid-2000s. Oh God, is this what qualifies you as "old" now

Kids these days, eh? Lawn, etc.

Re: Correctness and composability bugs in the Julia ecosystem

#294

Post by renox »

StefanKarpinski wrote: post #222
KenoFischer wrote: post #108

So this one is a tough one for me, because Yuri has certainly spent significant time with Julia and I think he's a very competent programmer, so his criticism is certainly to be taken seriously and I'm sad to hear he ended up with a sour opinion. There's a lot of different issues mentioned in the post, so I'm not really sure what angle to best go at it from, but let me give it a shot anyway. I think there's a couple…

The big language design problem that I think this post highlights is that the flip side of Julia's composability is that composing generic code with types that implement abstractions can easily expose bugs when the caller and the callee don't agree on exactly what the abstraction is. Several of the bugs that Yuri reported are a very specific case of this: there's a lot of generic code that assumes that array indexing…

I remember reading a long time ago about the 1-based array and the offset-array 'kludge'.

My first thought was they should have replicated Ada's design instead, my second thought I hope that they have a good linter because putting arbitrary offset implementation in a library is a minefield.

I don't claim to be especially smart: this is/was obvious.. Unfortunately what isn't obvious is how to fix this issue and especially how to fix the culture which produces this kind of issue..

Re: Correctness and composability bugs in the Julia ecosystem

#295

Post by grumpyprole »

adgjlsfhk1 wrote: post #286
grumpyprole wrote: post #273

Earlier quoted context omitted.

This is not true. For example, the issue regarding custom index ranges causing silent data corruption (6 examples) could be fixed with static types. Look how many of the other bug reports contain the phrase "does not check for" or refer to specific primitive types.

it is only fixed with static typing and no generics (i.e. C/Fortran). If you have a generic array supertype, a statically typed language would let you write exactly the same bug.

If one has a different type or trait for unusual and normal range indices, then the signature for the procedure that assumes indexing from 1 can be written to disallow other starting indices.

Re: Correctness and composability bugs in the Julia ecosystem

#296

Post by pron »

jolux wrote: post #282
pron wrote: post #278

Earlier quoted context omitted.

> People get along fine without this feature, therefore there’s no sense in implementing it. As someone whose job is to add new features to a programming language, that's never been my argument. > But it cuts the other way, or we’d all still be using assembly High-level languages were satisfactorily shown to be more productive than Assembly. I don't claim that no innovation works, just that not all do, and certainly…

> As someone whose job is to add new features to a programming language, that's never been my argument. I’ve definitely seen you argue along the lines of “it hasn’t been implemented in Java, therefore nobody uses it and we can’t tell if it’s a good idea or not” before. Forgive me for assuming this followed from that. > You can't come up with a claim, try and fail to support it time and again, and keep asserting it as…

> I’ve definitely seen you argue along the lines of “it hasn’t been implemented in Java, therefore nobody uses it and we can’t tell if it’s a good idea or not” before.

You have not seen me argue anything along those lines. I have, however, said the converse, that we try not to adopt features in Java until they've proven themselves elsewhere.

> However, in terms of eliminating patterns which are literally never correct, like dereferencing null pointers, violating resource lifetimes, or calling methods that don’t exist, static typing can in fact eliminate those patterns.

But the implication is reveresed! From A => B, i.e. types prevent certain bad things, you're concluding B => A, i.e. if you don't want those bad things then you should use types. That simply does not follow.

> This is an obvious advantage, and on some level I don’t really care if it contributes to formal correctness or not because it would make my job easier.

I wouldn't dare imply that types don't have certain important advantages, but that doesn't support the specific claim that types generally and significantly improve correctness — which many have tried to show and failed — and it certainly doesn't support the much stronger claim that if you want to improve correctness, the most effective way to do it is to use types.

Re: Correctness and composability bugs in the Julia ecosystem

#297

Post by grumpyprole »

DNF2 wrote: post #289
grumpyprole wrote: post #273

Earlier quoted context omitted.

This is not true. For example, the issue regarding custom index ranges causing silent data corruption (6 examples) could be fixed with static types. Look how many of the other bug reports contain the phrase "does not check for" or refer to specific primitive types.

How would static types help with that? Whether your indexing range starts with zero or one or something else isn't necessarily encoded in the type domain. `1:length(A)` is just a range of `Int`s.

Why not encode the starting offset into the type domain? Or at least distinguish between normal and unusual. Then the function signature can restrict to 1-offset arrays if that is what it assumes internally.

Re: Correctness and composability bugs in the Julia ecosystem

#298

Post by wnoise »

CRConrad wrote: post #277
wnoise wrote: post #161

Earlier quoted context omitted.

You're saying no while explaining my reasoning. What index to start with only strongly matters when the indexes have semantics. Otherwise you should just treat it as an opaque index, i.e. eachindex(), keys(), etc. In math when there are semantics, the indices usually include 0. When not (vector components, matrix indices, etc), they usually (but not uniformly) don't. The one nice side-benefit of Julia's mistake in ad…

> What index to start with only strongly matters when the indexes have semantics. Which in everyday computing (as opposed to mathematics) they often do, and those cases are (most?) often, in human terms, much more natural to start from 1: "I have an array of N elements. The first of a bunch of things is thing number one, and the last of N things is thing number N." Hence: N_things: Array[1-N] of Thing; for i := 1 to…

> often, in human terms, much more natural to start from 1

This meaning of natural is highly cultural dependent. It took the Greeks a startlingly long time to accept that one was a number (because it's a singleton), much less zero. I do not e.g. want arrays that can't have length one, because they have to be containing a number of things.

> No need to build anything new;

Well, no, not "new". Arrays with arbitrary bounds is a well-trod path. But they still had to make it work in Julia: CartesianIndices, LinearIndices, and overloading of "begin", and "end" keywords, etc. And the radical dependence on multimethod dispatch meant they couldn't quite just reuse existing work from other languages.

Re: Correctness and composability bugs in the Julia ecosystem

#299

Post by CRConrad »

buescher wrote: post #195
clarle wrote: post #43

Earlier quoted context omitted.

Genuinely curious (I don’t work with low level systems day-to-day), but why would C/C++ be better for safety? I would imagine that the risk of bugs related to memory access and data racing would be pretty high, compared to a higher-level language or something like Rust that focuses on safety.

Tooling and experience. In principle, there are other languages that might be better than C or C++ for safety-critical software. You know all the hoary jokes about the difference between theory and practice?

> You know all the hoary jokes about the difference between theory and practice?

Only in theory; do you have a practical example?

Re: Correctness and composability bugs in the Julia ecosystem

#300

Post by CRConrad »

FabHK wrote: post #292
CRConrad wrote: post #288

Earlier quoted context omitted.

But floors of multi-storey buildings are a pretty unique exception in the real world in having a characteristic where zero -- the number of stairs you need to climb from the ground floor -- has an actual tangible meaning (on the ground floor). How many other such examples can you (editorial you; anyone) come up with? Not many, I'd bet.

I thought you'd be wrong, and immediately came up with: - Hours after midnight. The (non-anglosaxon) watch goes from 0:00 to 24:00 (the latter is useful for deadlines: The proposal must be submitted by Friday, 24:00 (which coincides with Saturday, 0:00)). Then I cheated and looked up Wikipedia ( https://en.wikipedia.org/wiki/Zero-based_numbering#Other_fie... ) - and whoops, that's basically it!

Yeah, duh, totally forgot that one.

And I should know; the alarm clock on my windowsill says 0:54 right now.

Post reply on HN