Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correctness and composability bugs in the Julia ecosystem

yuri.is

Post reply on HN
Print view View original

161–170 of 419 posts

Re: Correctness and composability bugs in the Julia ecosystem

#161

Post by wnoise »

sdfhdhjdw3 wrote: post #116
wnoise wrote: post #61

Earlier quoted context omitted.

Starting with 0 is quite common in series, e.g. Taylor, Fourier, Chebyshev expansions, etc.

No... in those cases you're starting with 0 because that's the lowest exponent of a polynomial.

You're saying no while explaining my reasoning.

What index to start with only strongly matters when the indexes have semantics. Otherwise you should just treat it as an opaque index, i.e. eachindex(), keys(), etc. In math when there are semantics, the indices usually include 0. When not (vector components, matrix indices, etc), they usually (but not uniformly) don't.

The one nice side-benefit of Julia's mistake in adopting 1-based indexing is that it provided an extra impetus to build machinery to handle arbitrary indexing, though too much code still doesn't work correctly, and code still gets written to only handle 1-based arrays.

Re: Correctness and composability bugs in the Julia ecosystem

#162

Post by FabHK »

rashidrafeek wrote: post #131
SemanticStrengh wrote: post #77

Earlier quoted context omitted.

any tutorial/blog on what make julia compositionability special vs othe languages? Is there a relation with multiple dispatch or delegation?

Yes. Its a side effect of multiple dispatch being the core paradigm of the language. See Stefan Karpinski's talk about it: https://www.youtube.com/watch?v=kc9HwsxE1OY

The title of Stefan's talk is great: The Unreasonable Effectiveness of Multiple Dispatch. He gives a nice example of composability: how you can throw a new type into an existing algorithm and it just works.

Re: Correctness and composability bugs in the Julia ecosystem

#163

Post by cozzyd »

harpiaharpyja wrote: post #63
tqaky wrote: post #19

Quoted post unavailable.

I'm guessing you've never used Python for a serious project before, because your statement is incorrect. Aside from the fact that Python is memory safe, my experience has been that it is far easier to avoid logic errors in Python than in C/C++. Equivalent Python code is shorter and less noisy than C/C++, and Python makes it easier to create succinct and simple to use abstractions. The result is that code is far easie…

In my experience it's much harder to write Python that won't crash than C/C++ that won't crash. With Python, rarely taken code paths can have very dumb errors in them (e.g. accidentally introducing a new variable with a different name than the variable you wanted or typoing a method name, or passing arguments with the wrong type) that would get caught by a C/C++ compiler but won't be caught in Python until the crash happens some months later. Typically in "reliable" code you're not going to do much dynamic allocation anyway, at least past the initialization stage (because memory fragmentation!) so memory errors aren't that common anyway.

Yes, probably Rust and Ada provide even more guarantees...

Re: Correctness and composability bugs in the Julia ecosystem

#164

Post by wnoise »

pdeffebach wrote: post #112
wnoise wrote: post #87

Earlier quoted context omitted.

That comment doesn't say all bugs have been fixed, or even quickly fixed. When I check on the posted links, many are in fact still open, e.g. https://github.com/JuliaStats/Distributions.jl/issues/1253 https://github.com/JuliaStats/StatsBase.jl/issues/642 https://github.com/JuliaStats/StatsBase.jl/issues/616 https://github.com/JuliaLang/julia/issues/39385

I don't get these complaints about `sum!(a, a)`. Sure it's a bit of a footgun that you can overwrite the array you are working with. This doesn't rise to a "major problem" of composability. The histogram errors seem annoying though. Hopefully they can get fixed.

Sure, it's unsurprising that it produces unexpected results, but there are actually semantics that should be expected. The problem is that implementing those semantics correctly for all cases is hard, because aliasing. Same issue that e.g. memcpy() vs memmove() have.

Re: Correctness and composability bugs in the Julia ecosystem

#165

Post by FabHK »

ThenAsNow wrote: post #91
QuackingTheQ wrote: post #52

I've spent a lot of time developing large computational codebases in Julia, and I think the most insidious of these issues is a product of no formal way of enforcing interfaces. Using one of the common packages to build a trait system and add some sort of guarantee that all the right methods are implemented for a given trait simplifies maintenance dramatically. This doesn't catch mathematical bugs, but those crop up…

I agree with the kernel of your point here, but also with the author of the article when he says "But systemic problems like this can rarely be solved from the bottom up, and my sense is that the project leadership does not agree that there is a serious correctness problem. They accept the existence of individual isolated issues, but not the pattern that those issues imply." My impression is that the Julia core devs…

Sounds like Julia needs a Snow Leopard/Mountain Lion/High Sierra release - no new features, just cleaning things up...

Re: Correctness and composability bugs in the Julia ecosystem

#166

Post by kazinator »

ChrisRackauckas wrote: post #44

Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from tho…

> I agree, code should never do that. It should be `eachindex(A)`

Will that generate the same code as "i in 1:length(A)"?

Maybe whoever wrote that didn't believe so at least, or perhaps didn't find it so at the time.

The reason @inbounds would have been used is performance, so that's likely why the for loop header was written that way?

Re: Correctness and composability bugs in the Julia ecosystem

#167

Post by pdeffebach »

wnoise wrote: post #164
pdeffebach wrote: post #112

Earlier quoted context omitted.

I don't get these complaints about `sum!(a, a)`. Sure it's a bit of a footgun that you can overwrite the array you are working with. This doesn't rise to a "major problem" of composability. The histogram errors seem annoying though. Hopefully they can get fixed.

Sure, it's unsurprising that it produces unexpected results, but there are actually semantics that should be expected. The problem is that implementing those semantics correctly for all cases is hard, because aliasing. Same issue that e.g. memcpy() vs memmove() have.

What semantics are expected in other languages? This seems solidly in the realm of undefined behavior as far as I can tell.

Re: Correctness and composability bugs in the Julia ecosystem

#168

Post by snicker7 »

Sukera wrote: post #41

Most of these seem to be about packages in the ecosystem (which, after clicking through all links, actually almost all got fixed in a very timely manner, sometimes already in a newer version of the packages than the author was using), not about the language itself. Other than that, the message of this seems to be "newer software has bugs", which yes is a thing..? For example, the majority of issues referenced are spe…

"known to be incompatible"

Known to whom? People who regularly participate in the Julia forum/chat? Julia's composability relies on people agreeing on unwritten rules and standards.

In other languages, such incompatibilities are caught by the compiler. Even in other dynamic languages like Python or Javascript, it is now considered best practice by many to annotate types whenever you can. Like Julia, Haskell is also composable. Unlike Julia, it does not need to sacrifice correctness.

Re: Correctness and composability bugs in the Julia ecosystem

#169

Post by FabHK »

Rayhem wrote: post #83
coldtea wrote: post #71

Earlier quoted context omitted.

> It very, very much is. Polynomials all start at a zero "index" Notice how you had to put index in quotes. Because it's not an index, it's the degree of each polynomial term, which is a power.

Notice how you're ad homenim-ing the structure of the argument and not the argument itself? I do not at all see how putting quotes around that word invalidates the argument. I did so because mathematical literature doesn't refer to it as an index (rather as a degree as you mentioned), but it very much does index each monomial. There are an infinite number of index sets for each polynomial -- just as i can index the i…

GP was addressing the argument. Using the word "you" in the reply does not make it ad hominem.

(For more than you ever wanted to know about this, The Ad Hominem Fallacy Fallacy covers it exhaustively and entertainingly:

https://laurencetennant.com/bonds/adhominem.html )

Re: Correctness and composability bugs in the Julia ecosystem

#170

Post by leephillips »

nightpool wrote: post #103
ChrisRackauckas wrote: post #44

Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from tho…

I'll be honest, based on my experience with Julia, this makes me more worried about using e.g. libuv in production systems now, not less. I understand your opinion that "The easier it is to look at the code, the easier it is to find issues with it", but I don't think that has anything to do with the fact that `prod((Int8(100), Int8(100)))` and `prod([Int8(100), Int8(100)])` disagree, because someone decided to specia…

Just to clarify, the prod() bug you mention was fixed about a year ago.
Post reply on HN