Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

71–80 of 136 posts

Re: Statistics with Julia [pdf]

#71

Post by eigenspace »

mlevental wrote: post #47
caiocaiocaio wrote: post #6

Julia looked interesting to me, so I tried 1.0 after it came out. I have a oldish laptop (fine for my needs), and every time I tried to do seemingly anything, it spent ~5 minutes recompiling libraries or something. So I've been waiting newer versions that hopefully stop doing that, or for me to buy a better computer.

my bigger problem is how unstable all of the apis are. every single time i try to follow a guide/tutorial i get compilation errors because packages have shifted.

Now that 1.0 is out, APIs have stabilized a ton, even in the package ecosystem. But depending n your stability needs, packages might still be changing too fast.

I’d say for most people, there’s so much great progress and improvements happening that the breakages are well worth it.

Re: Statistics with Julia [pdf]

#72

Post by [deleted] »

kgwgk wrote: post #55
mruts wrote: post #40

Earlier quoted context omitted.

I’m not sure what Python’s goals are to be honest. It seems to me that the language is outclassed in every way by better, more consistent, more powerful, and more performant languages. Python programmers seem content implementing the same things over and over again. Like, for example, flattening a list/monad. List of things python doesn’t have but should: pattern matching, multi-line lambdas, more data structures (lo…

"In a 1999 report, Van Rossum highlighted the following as his goals for Python: It should be an easy and intuitive language, just as powerful as major competitors. It should be open source, so anyone can contribute to its development. Its code should be understandable as plain English. It should be suitable for everyday tasks, allowing for short development times." https://www.computerhistory.org/fellowawards/hall/g…

[deleted]

Re: Statistics with Julia [pdf]

#73

Post by improbable22 »

Merrill wrote: post #69

In section "1.2 Setup and Interface" there is a very short description of the REPL and how it can be downloaded from julialang.org, as well as a much longer description of JuliaBox and how Jupyter notebooks can be run from juliabox.com for free. Although JuliaBox has been provided for free by Julia Computing, there has been discussion that this may not be possible in the future. However, Julia Computing does provide…

No, I think you should simply download the ordinary version. Jupyter, Juno, etc. are easy enough to install locally. I forget the precise details, but I think JuliaPro comes with certain versions of packages, and it's less confusing just to get the latest of what you need (using the built-in package manager).

JuliaBox (and https://nextjournal.com/) are cloud services, but if you have a real computer and want to do this for more than a few minutes, just install it. (There's also no need for virtualenv etc.)

Re: Statistics with Julia [pdf]

#74

Post by j88439h84 »

eigenspace wrote: post #67
j88439h84 wrote: post #60

Earlier quoted context omitted.

> Julia has static typing. Julia has its own type system, which doesn't conform to the traditional static/dynamic divide. But AFAIK it doesn't have a compile-time type checker like mypy to help me catch type errors early.

Yeah, Julia’s type system is really just not designed for the sort of error catching patterns people seem to want static type systems for. Instead, Julia’s type system is designed for enabling multiple dispatch which I’d argue is a much greater boon than the dubious claims of error catching due to static type systems. However, you can simulate static typing with the @inferred macro from Test.jl. It will throw an erro…

I'm not sure what you mean about 'dubious claims of error catching'. Mypy catches my errors all the time. Sometimes I have to fight with it, but it's definitely worthwhile if I care about having a working program. And when I don't care, I can just ignore any overzealous warnings.

Re: Statistics with Julia [pdf]

#75

Post by ChrisRackauckas »

Cybiote wrote: post #24
ChrisRackauckas wrote: post #3

This is a very good resource. The one thing I would ask is that I would like to see examples of using DifferentialEquations.jl when you get to the section on dynamical systems, especially when doing discrete event simulation and stochastic differential equations. I opened an issue in the repo and we can continue discussing there (I'll help write the code, I want to use this in my own class :P)!

I agree it's a wonderful resource. Which is exactly why I disagree with your suggestion. The book is uncommonly clear in how it explains fundamentals and bringing in such a powerful library ends up moving quite a bit away from that. It will no longer be just about the fundamentals of Julia on one hand and on the other, the algorithms will no longer be implementing language invariant. Losing that invariance IMO makes…

I would say calling an ODE solver is pretty fundamental to a lot of real scientific workflows, but I am pretty biased on that.

Re: Statistics with Julia [pdf]

#76

Post by jointpdf »

anthony_doan wrote: post #64
jointpdf wrote: post #31

Earlier quoted context omitted.

I think Julia has a cleaner focus on scientific and mathematical computing than either R or Python (both for performance and understanding). i.e. the language is designed in such a way that corresponds more directly to mathematical notation and ways of thinking. If you’ve been in a graduate program that’s heavily mathematical, where you spend equal time doing pen and paper proofs and hacking together simulations and…

> R is too quirky to fulfill this niche I'd like to offer a counter point or add on to this. It's quirky enough to have many packages backed by some expert statistician. I hope Julia get to be successful in this regard too.

The way I wrote that comes off as more dismissive than I intended. I think it’s quirky in the sense that there is a wide variance in styles of accomplishing things in (base) R, so something that appears perfectly natural to me can look foreign to someone else. I think this is partly the user base and partly the language itself, and of course the two are interdependent. To me, it’s a joy to write R code because of it’s flexibility and power, but I often have dreaded sharing it with others (especially as a beginner). It’s easy to look at someone else’s R scripts and think “this is horrifying”. By the way, this is referring more to scientific/statistical workflows—for more general purpose data science in R, the Tidyverse (or even just the pipe operator %>% around which the Tidyverse is built) goes a long, long ways towards helping people write expressive but readable code.

By contrast, Python feels a bit too rigid/standardized. Everyone’s code looks like it was copy+pasted from a book of truth somewhere. This is good for sharing and engineering, not as good for expressing mathematical ideas.

So whereas R has evolved organically over decades and Python is for everyone (and alternatives like MATLAB or SAS are first and foremost software for industry rather than languages), Julia seems to be thoughtfully purpose-built to be a modern language for numerical/scientific computing. It polishes off the rough edges and blends some of the best features of each language. Again, this is just an impression from someone who already thinks in R but is learning both Python/Julia.

More to your point, maybe Julia is at a stage of development where it’s good for both students (for developing computational and mathematical thinking) and experts (for slinging concise but performant code), but not yet the rank-and-file users looking to just get things done.

Re: Statistics with Julia [pdf]

#77

Post by kbd »

mruts wrote: post #61
kbd wrote: post #54

Earlier quoted context omitted.

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

The classic example is getting the last element of an array. With 1-based indexing the length of the array is the index of the last element. It has a nice symmetry to it. Also I find it elegant that for 1-indexing that the start and end value for slices are both inclusive, instead of the first one being inclusive and the last being exclusive. Also, isn’t it just weird that the index of an element is one less than it’…

> The classic example is getting the last element of an array.

Good point, in Python I don't notice that the last element is arr[len(arr)-1] because Python provides arr[-1]. I think in general your point is that it's natural for the nth element to be arr[n].

> The reason for zero indexing is historical, related to pointer offsets.

There is that, but Dijkstra's paper makes the case from first-principles that the closed, open interval of [0,n) for sequences is the most appropriate.

> with 1-indexing I can multiply numbers by arrays... 3 x 1 is three...

Sorry, I don't understand this. It makes sense that the point I don't understand is probably most-related to why Julia chose its indexing scheme and why Matlab et al. do the same.

> One nice this about 0-indexing is that I can slice a list in half with the same midpoint.

Yeah, arr[0:index] + arr[index:len(arr)] is the full list. And to your point earlier ("if I take the first nth elements of a list"), len(arr[:n]) == n seems natural.

Edit: I've been trying to formalize why Python's indexing scheme, along with its negative-indexing, is optimal (slight pseudocode):

    l = ['a','b','c']
    n = len(l)
    i = -n
    while i
prints "a b c a b c". That code makes no reference to any bound but 'n', nor any constants (1,0) or offsets, yet it iterates over the list twice through its range (first negative indices then positive).

Re: Statistics with Julia [pdf]

#78

Post by hsaliak »

j88439h84 wrote: post #52
mruts wrote: post #42

Earlier quoted context omitted.

Julia has static typing. I wouldn’t go that far and say Python is suitable for large programs. It’s clearly not. Working on a large python code base is hell.

Old Python, before mypy, attrs/dataclasses, etc., is a pain. Nowadays with modern tooling, it's terrific.

this. I find it hilarious that people keep comparing a python release from 2010 with something that was released in O(months). Modern python - mypy, attr/dataclasses has come a long way. Numba has also come a long way since it's initial release.

Re: Statistics with Julia [pdf]

#79

Post by j88439h84 »

superdimwit wrote: post #48
kgwgk wrote: post #27

Earlier quoted context omitted.

> Julia is everything python could have been The goals of Python were quite different from the goals of Julia.

In my opinion, it's an unfortunate accident that Python became popular for numerical / data-ey workloads. It's good for some things, but fast low-overhead loopy code is definitely not one of them!

CPython, yeah :\

I wish the pypy team had finished numpypy. Then fast numerical programs could be written in python instead of relying on all kinds of C extensions and stuff. Python would be great for numerics then.

Re: Statistics with Julia [pdf]

#80

Post by cauthon »

superdimwit wrote: post #38

I'd really recommend anyone doing mildly numerical / data-ey work in python to give Julia a patient and fair try. I think the language is really solidly designed, and gives you ridiculously more power AND productivity than python for a whole range of workloads. There are of course issues, but even in the short time I've been following & using the language these are being rapidly addressed. In particular: generally le…

Are all the plotting/visualization options still half baked?
Post reply on HN