I tried julia last year, and it was a nightmare of version skew. Has it improved in that regard at all?
Some Insights from a Julia Developer
151–160 of 241 posts
Re: Some Insights from a Julia Developer
#152Earlier quoted context omitted.
If you read the OP again carefully, you'll see that you can use offset arrays for every array you make and suffer no performance penalty, because the offset is compiled away. And the development overhead is a single library import call; I replace core language data-structures all the time in any language for features I want, this is no different.
See my other comment in the thread for the thing I've always wondered: How can you compile away an arbitrary calculation that might need to be made at run time? (assuming we're not just arguing over arbitrary-array indexing to input predefined constants at the REPL/code level).
Could it be that the offset calculation to find the start of the array is needed in any case?
Re: Some Insights from a Julia Developer
#153I'm wondering about this part: "using this strategy Julia actually can produce static binaries like compiled C or Fortran code." Is this something that works now, something planned, or just speculation?
It does work. It is not user friendly just yet, and compiler work needs to be done - but yes, you can get binaries and shared libraries today.
Re: Some Insights from a Julia Developer
#154Earlier quoted context omitted.
Any time 1-based indexing is mentioned as a shortcoming of Julia reminds me of PG's 'Blub paradox' [1]. > As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power contin…
I've seen the claim that Julia has offest-arrays at essentially no additional cost a few times now, and I'm just not buying it. As in, I don't see how its possible without stretching the meaning of "no additional cost". I'm familiar with displaced arrays and the like in Common Lisp, so i get that you can do offsets and things without using much additional memory, and i feel its a small jump from that to arbitrary ind…
This is easily testable:
julia> using BenchmarkTools
using OffsetArrays
A = rand(1000)
O = OffsetArray(A, 0:999)
function naive_sum(A)
s = zero(eltype(A))
@unsafe for i in eachindex(A) # @unsafe will eventually be folded into @inbounds
s += A[i]
end
return s
end
naive_sum (generic function with 1 method)
julia> @btime naive_sum($A)
1.025 μs (0 allocations: 0 bytes)
514.0960505118594
julia> @btime naive_sum($O)
1.023 μs (0 allocations: 0 bytes)
514.0960505118594
Re: Some Insights from a Julia Developer
#155Sometimes, in my darker moments, I have the terrifying thought that one of the reasons that many users like R and made it popular (apart from the historical context of its now many libraries and being the main free version of statistical software), is specifically that it isn't robust and sensibly designed from a programming/analytical perspective. You can download a package, type in a preset command on a preset thin…
I think your fears are misplaced.
R users love R because of ease of use. R does sometimes ignores ugly corner cases in favor of that ease of use (though I'm skeptical this damages the validity of the answer in anything like 5% of cases), but that's a side effect.
sklearn and pandas are great, but you still simply have to be a programmer to use them, or at least much closer to a programmer than many statisticians want to be. Allowing non-programmers to do things like
data This omission of most quoting, the magic inference of column names, etc etc all makes R much easier to use.
Re: Some Insights from a Julia Developer
#156Earlier quoted context omitted.
> To quote wikipedia on Zero-based "Zero-based numbering or index origin = 0[1][2] is a way of numbering in which the initial element of a sequence is assigned the index 0, rather than the index 1 as is typical in everyday non-mathematical/non-programming circumstances." 1 Based 0 = 1 1 = 2 Zero based 0 = nothing 1 = 1 Seem important for mathematics.
Also head and tail in Pandas is 1 based! That drove me crazy. print(users[users.age > 25].head(3)) Returns 3 elements and not 4!
Is it possible that the argument to `head` is the _number_ of elements to return, and that the choice of indexing therefore doesn't enter the picture?
Re: Some Insights from a Julia Developer
#157Earlier quoted context omitted.
The improvements are not minor they are massive. Citing rust shows that the advantage of Julia has not been explained well enough. Julia allows you to write as performant code as Rust with a much smaller investment in learning. You cite your concern for spending time learning something new. That makes no sense considering the high learning curve and complexity of Rust compared to Julia. Julia is quite fast to learn a…
Have a citation on the performance vs Rust? I'm skeptical any GC'd language can approach C/Rust unless they have explicit mechanisms to do data layout for using the cache/prefetcher to the fullest degree.
The language page shows micro benchmarks that Julia is comparable to C in performance, when optimized. I've seen various users of the language report similar experience for packages they've built. Since C and Rust are similar in performance I would assume Julia is close to Rust in performance.
You can also easily verify this yourself by looking at assembly code dumps for JITed Julia functions. You can see that the amount of code would be comparable to that of C for many common scenarios.
As for cache and data layout. That GC'd language can't control data layout and cache usage is simply wrong. Best known example is probably Go. You can nest structs to create contiguous blocks of memory in Go, just like in C. You can also create arrays of structs as contiguous blocks of memory.
Just because Java and C# doesn't work that way, doesn't mean all GC'd languages are like that.
A struct of concrete types in Julia is just like a struct in C in the way it is packed in memory. An array of structs in Julia or a nesting of structs will have similar memory layout as in C.
So I know this sounds really odd, but Julia despite being a dynamic language thus allows more optimal memory layout than a statically typed languages such as Java.
What is frustrating with promoting Julia, is that it simply sounds way too good to be true, so people dismiss the language before they have actually read up on it. I advice you to read up a bit more on the details of how the language works. It will then become clear why it can pull off stuff which seems far too good to be true.
Re: Some Insights from a Julia Developer
#158Too bad they somehow thought it was a good idea to make the syntax resemble MATLAB of all languages. Perhaps most of the nausea inducing warts could be worked around with some kind of transcompilation, although some semantic issues, such as one-based indexing, would remain. It'll be a sad day if Julia starts to get such popularity that high quality libraries will be Julia-only.
I really don't get the indexing religious wars. I "grew" up on 0-indexed languages, C/C++, assembly, Python, Ruby, Java, Objective-C, Swift etc.
Yet I've never felt problems using 1-based languages whether Lua or Julia.
To me I just switch my mental mode to think I am doing math. I am used to mathematics using 1-indexing. And I am not a physicist or mathematician so it is not like I am steep in this tradition.
Of all things one might object to in a language, I don't grasp this carries so much weight for many people.
Re: Some Insights from a Julia Developer
#159Earlier quoted context omitted.
Not sure how much performance matters here. You're just farming it off to CPLEX or Gurobi anyway. If performance really matters, you'll do this part in C++ anyway.
Unless you're solving toy problems, it matters quite a bit. In mathematical optimization, constructing the problem tends to be just as expensive as solving the problem is – sometimes more so. The existence of expensive commercial systems like AMPL and GAMS that only exist to express optimization problems demonstrates that this is a non-trivial issue that people are willing to pay money for. Using C++ APIs to solvers…
Solver licenses are extremely expensive (~100k in production), so switching solvers isn't very common. Going to C++ helps a bit with I/O before it gets to the solver if you really need it, but a lot of people do just fine with things like AIMMS which is a proprietary high level modeling language similar in performance to Python. Maybe Julia + JUMP is nice if you want to remove I/O performance barriers, but don't want C++ pain & more flexibility, but I don't see JUMP being talked about much in the industry currently. If you don't mind me asking, what is your experience in this area and what domain are you working in?
Re: Some Insights from a Julia Developer
#160Yeah, the language design of julia is brilliant (multiple dispatch, typing, llvm use, zero-cost abstractions, @code_native to see why your code is slow). This allows you to write fast code in julia, which is impossible in python (you can call into very fast C/Fortran libraries with nice bindings, though). On the other hand, I really hate the syntax. One-based array indexing (ok, minor), blocks ending with "end", and…
I greatly prefer Julia syntax. `end` makes the code blocks stand out more easily than }. I find it easier to see the indentation at a glance than when just dealing with a single thin character.
Also I like that Julia prefers shorts words over special characters. C/C++ use far too many special characters.
That might run counter to my delight at unicode support. I think it is quite nice to be able to write mathematical code using the same symbols as used in mathematics. I use the Julia REPL to write this where you got latex completions to get unicode characters.
"Second non-cosmetic problem is that the language documentation is atrocious, both from a completeness and pedagogical viewpoint. Pyhton is again the ideal to aspire to."
I could not disagree more. Here is an account of Julia vs Python from mostly a pedagogical point of view: https://medium.com/@Jernfrost/python-vs-julia-observations-e...
A few points. Documentation is often easier to read. E.g. look at the example of the `print` function.
Function usually have more sensible names in Julia and you don't have to guess which package they are hidden in for common things such as working with strings, paths and arrays. Part of this is due to Julia multiple dispatch which allows reusing the same function name for related functionality, while python is forced to invent unique function names too often. Even when it doesn't make sense.
I'd say Julia adheres to the Python zen of least surprise. Checking if a collection is empty, can be done with `isempty()` which is similar to a number of other languages. Python in contrast treats empty lists as boolean objects which are false when empty. How is that obvious?