Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

31–40 of 136 posts

Re: Statistics with Julia [pdf]

#31

Post by jointpdf »

bdod6 wrote: post #28

Can someone explain how this is more powerful than someone use an Python/R based workflow? E.g., I currently use a combination .ipynb, python scripts, and RStudio and this feels like it covers everything I need for any data science project.

I think Julia has a cleaner focus on scientific and mathematical computing than either R or Python (both for performance and understanding). i.e. the language is designed in such a way that corresponds more directly to mathematical notation and ways of thinking. If you’ve been in a graduate program that’s heavily mathematical, where you spend equal time doing pen and paper proofs and hacking together simulations and such (and frantically trying to learn a language like R/MATLAB/Python while staying afloat in your courses), you’ll appreciate the advantage of this. To my eyes, Python is too verbose and “computer science-y” and R is too quirky to fulfill this niche (I say this as someone that bleeds RStudio blue, and enjoys using Python+SciPy). I don’t think Julia is aimed at garden-variety / enterprise data science workflows. Caveat—I’m not a Julia user currently, so this is sort of a hot take.

The “Ju” in Jupyter is for Julia, so it’s designed to be used as an interactive notebook language also. The Juno IDE is modeled after RStudio.

Re: Statistics with Julia [pdf]

#33

Post by zmk_ »

aurelian15 wrote: post #16
SolarNet wrote: post #12

Earlier quoted context omitted.

This is a core part of the design. It's part of why Julia is so useful for scientific computing, where one often has a large job that will require a lot of processing time, such that it is worth it to do an intensive JIT cycle every-time. And part of that is the analysis to take python-esque code and turning it into C levels of performance.

I just looked into Julia (1.1) for scientific use (simulation of very simple dynamical systems) a few days ago. I have to admit that by the end of the day I was surprisingly frustrated. I felt that type annotations were insufficient (one of the reasons to move away from Python); in particular, I didn't find a way to specify statically sized array types as you can do with Eigen, a feature that I find incredibly useful…

About Gadfly. If you were plotting it in a notebook then the default is to make it an interactive figure. In that case a lot of overhead comes from the browser trying to render it. You can explicitly invoke e.g., draw(SVG(10cm,10cm), plot(...)) to make it much faster.

Re: Statistics with Julia [pdf]

#34

Post by j88439h84 »

jointpdf wrote: post #20

This looks like a good reference for the fundamentals of both statistics and Julia, as claimed. I have a small critique, since the authors asked for suggestions. The format for the code samples goes like (code chunk —> output/plots —> bullet points explaining the code line-by-line). This creates a bit of a readability issue. The reader will likely follow a pattern like: (Skim past the code chunk to the explanation —>…

I'm not sure how that allennlp site is doing it, but source is here: https://github.com/allenai/allennlp/blob/b0ea7ab6be2787495fa...

Re: Statistics with Julia [pdf]

#35

Post by j88439h84 »

mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

Does it have a type checker like mypy? Python's protocols (https://www.python.org/dev/peps/pep-0544/) are "structured enough for large programs".

Re: Statistics with Julia [pdf]

#36

Post by cwyers »

dlphn___xyz wrote: post #26

whats the selling point with Julia? why would i use it over something like R?

In R, most of the high performance code isn't written in R, it's written in Fortran or C or C++ (R has really good C++ integration via Rcpp). Python has something similar. The value prop of Julia is supposed to be that you have a language flexible enough to do the high-level stuff you'd normally do in R/Python, plus the ability to write high-performance code without having to drop into another language.

I remain skeptical that this solves a lot of real-world problems (I know a lot of users of R/Python who never need to resort to writing their own C/C++ code), but that's the sales pitch.

Re: Statistics with Julia [pdf]

#37

Post by orbifold »

aurelian15 wrote: post #16
SolarNet wrote: post #12

Earlier quoted context omitted.

This is a core part of the design. It's part of why Julia is so useful for scientific computing, where one often has a large job that will require a lot of processing time, such that it is worth it to do an intensive JIT cycle every-time. And part of that is the analysis to take python-esque code and turning it into C levels of performance.

I just looked into Julia (1.1) for scientific use (simulation of very simple dynamical systems) a few days ago. I have to admit that by the end of the day I was surprisingly frustrated. I felt that type annotations were insufficient (one of the reasons to move away from Python); in particular, I didn't find a way to specify statically sized array types as you can do with Eigen, a feature that I find incredibly useful…

Julia is what happens if you let amateurs develop a compiler. The few times I’ve tried it produced gigabytes worth of stuff super slowly. The majority of packages are half backed, the only way to discover any type error is to let the program run, which coupled with multi method dispatch and hellishly slow compile times for trivial amounts of code makes the whole experience super unpleasant. Modern C++ plus some python feels more pleasant to work with (lightning fast compiles).

Re: Statistics with Julia [pdf]

#38

Post by superdimwit »

I'd really recommend anyone doing mildly numerical / data-ey work in python to give Julia a patient and fair try.

I think the language is really solidly designed, and gives you ridiculously more power AND productivity than python for a whole range of workloads. There are of course issues, but even in the short time I've been following & using the language these are being rapidly addressed. In particular: generally less rich system of libraries (but some Julia libraries are state of the art across all languages, mainly due to easy metaprogramming and multiple dispatch) + generally slow compile times (but this is improving rapidly with caching etc). I would also note that you often don't really need as many "libraries" as you do in python or R, since you can typically just write down the code you want to write, rather than being forced to find a library that wraps a C/C++ implementation like in python/r.

Re: Statistics with Julia [pdf]

#39

Post by xvilka »

Note that Julia 1.2[1] is on the verge[2] of being released. Also, it is interesting to see the list[3] of GSoC and JSoC (Julia's own Summer of Code). A lot of projects target the ML/AI applications. Personally, I am waiting for proper GNN support[4] in FluxML, but seems not much interest in it.

[1] https://github.com/JuliaLang/julia/milestone/30

[2] https://discourse.julialang.org/t/julia-v1-2-0-rc2-is-now-av...

[3] https://julialang.org/blog/2019/05/jsoc19

[4] https://github.com/FluxML/Flux.jl/issues/625

Re: Statistics with Julia [pdf]

#40

Post by mruts »

kgwgk wrote: post #27
mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

> Julia is everything python could have been The goals of Python were quite different from the goals of Julia.

I’m not sure what Python’s goals are to be honest. It seems to me that the language is outclassed in every way by better, more consistent, more powerful, and more performant languages.

Python programmers seem content implementing the same things over and over again. Like, for example, flattening a list/monad.

List of things python doesn’t have but should: pattern matching, multi-line lambdas, more data structures (look at Scala for an example of what kind of data structures a standard library should provide), real threading, options, monads, futures, better performance, and more.

Post reply on HN