Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

111–120 of 136 posts

Re: Statistics with Julia [pdf]

#111

Post by kmundnic »

spacedome wrote: post #84
cauthon wrote: post #80

Earlier quoted context omitted.

Are all the plotting/visualization options still half baked?

I've found Plots.jl and PyPlots.jl to work well for most basic things, despite not always being entirely pleasant to use, for example the compilation time issue, but this should hopefully improve. The only real problem I had is that these are not quite sufficient for plots to be published in a paper, many visual tweaks you might want are broken or terribly documented, and I have to just use matplotlib or R. It is gen…

No matter the tool I use these days for plotting, I export it as a .tex file to use PGFPlots. matlab2tikz, matplotlib2tikz, and the savefig function in Plots.jl all do the job (with the pgfplots backend). This way you can tweak the figure in the final document, which I prefer. You can adjust all of the properties of the plot in Latex.

Re: Statistics with Julia [pdf]

#112

Post by locoman88 »

eigenspace wrote: post #71
mlevental wrote: post #47

Earlier quoted context omitted.

my bigger problem is how unstable all of the apis are. every single time i try to follow a guide/tutorial i get compilation errors because packages have shifted.

Now that 1.0 is out, APIs have stabilized a ton, even in the package ecosystem. But depending n your stability needs, packages might still be changing too fast. I’d say for most people, there’s so much great progress and improvements happening that the breakages are well worth it.

... there’s so much great progress and improvements happening that the breakages are well worth it.

This.

According to the fanboys, Julia, and its ecosystem, is production ready, but the reality is that it's far from it. They rushed to release v1.0 when there were still lots of sharp corners to cut and polish.

What the Julia community needs to do now is: 1. Admit that the language is not ready yet and that there are lots of room for improvement, and 2. Stop badmouthing Pyhton and R

Re: Statistics with Julia [pdf]

#113

Post by nikhilsimha »

eigenspace wrote: post #92
kbd wrote: post #54

Earlier quoted context omitted.

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

Shouldn’t Dijkstra’s paper be your 0th reference?

^ This is why I love hackernews!

Re: Statistics with Julia [pdf]

#114

Post by oivey »

SolarNet wrote: post #82
j88439h84 wrote: post #53

Earlier quoted context omitted.

What's the nuance? It's much faster?

Since neither of the others mentioned it. The nuance is the type system + multimethods. It's a gradually typed system that fully specializes code where it can (aided by the expressive power of multi-methods), and hence with some careful (or overkill) placement of types (and multimethods) it's easy to get large performance boosts with minor edits to one's code (rather than porting the whole thing to C which is the pyt…

I wouldn’t say that Julia is gradually typed in the same way that Cython or Numba is. In my experience you usually improve performance by ensuring that your functions handle different types generically. One example of that is making sure the compiler can infer the types of all your variables to something more specialized than the Any type. Another example is being careful to avoid accidental type changes by e.g. summing a Float32 with a Float64 literal.

As I’ve learned the language it’s become pretty easy to avoid those pitfalls even on initial implementations. That said, providing types in function signatures is still very useful for multiple dispatch and providing a more usable API in libraries.

Re: Statistics with Julia [pdf]

#115

Post by ChrisRackauckas »

aurelian15 wrote: post #16
SolarNet wrote: post #12

Earlier quoted context omitted.

This is a core part of the design. It's part of why Julia is so useful for scientific computing, where one often has a large job that will require a lot of processing time, such that it is worth it to do an intensive JIT cycle every-time. And part of that is the analysis to take python-esque code and turning it into C levels of performance.

I just looked into Julia (1.1) for scientific use (simulation of very simple dynamical systems) a few days ago. I have to admit that by the end of the day I was surprisingly frustrated. I felt that type annotations were insufficient (one of the reasons to move away from Python); in particular, I didn't find a way to specify statically sized array types as you can do with Eigen, a feature that I find incredibly useful…

The DiffEqTutorials has a tutorial which covers how to use statically-sized arrays to optimize the simulation of dynamical systems.

http://juliadiffeq.org/DiffEqTutorials.jl/html/introduction/...

Re: Statistics with Julia [pdf]

#116

Post by iamcreasy »

the_duke wrote: post #9
grzm wrote: post #8

Earlier quoted context omitted.

You can effectively bookmark submissions by using the "favorite" link or just upvoting. The submission will show up in your profile under "favorite submissions" or "upvoted submissions", respectively.

In addition, I hear that modern browsers support a ground-breaking functionality called "Bookmarks".

I would not rely on browse bookmarks too much. Recently I lost a large amount of bookmarks for google chrome sync overwriting my local copy. The bookmarks.bak file was missing too.

Re: Statistics with Julia [pdf]

#117

Post by iamcreasy »

ChrisRackauckas wrote: post #75
Cybiote wrote: post #24

Earlier quoted context omitted.

I agree it's a wonderful resource. Which is exactly why I disagree with your suggestion. The book is uncommonly clear in how it explains fundamentals and bringing in such a powerful library ends up moving quite a bit away from that. It will no longer be just about the fundamentals of Julia on one hand and on the other, the algorithms will no longer be implementing language invariant. Losing that invariance IMO makes…

I would say calling an ODE solver is pretty fundamental to a lot of real scientific workflows, but I am pretty biased on that.

I do not remember using much calculus other than usign it to pass the college courses. Can you point me to some resources that would teach me how to use calculus(or ODE if that's more interesting) to solve interesting problems?

Re: Statistics with Julia [pdf]

#118

Post by wodenokoto »

kbd wrote: post #54
mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

Looking at teaching materials, indexing in R hardly gets a mention. Student are told they get get the first element by a[1] and the twelfths element by a[12], and if you want 4th, 5th and 6th, you just ask for that range, a[4:6]

For python teaching this is almost a whole chapter, with people sharing cheat sheets and building graphics to show how slicing works what not. You don't see these things in R teaching materials.

I'm sure that for the implementation of algorithms, things might be easier with zero indexing, but for a user asking for element 4,5 and 6, 1-indexing is much, much easier on the user.

Re: Statistics with Julia [pdf]

#119

Post by pjmlp »

eigenspace wrote: post #90
iamed2 wrote: post #83

Earlier quoted context omitted.

Dijkstra's write-up is full of subjective aesthetic judgements that certain things are ugly. I personally don't find `1:0` for an empty sequence to be ugly, and I do find using `1:1` to refer to an empty sequence and `1:2` to refer to the sequence `{1}` to be ugly. I would encourage everyone to read over his reasoning and see if you agree with his aesthetic judgements.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

Because of the author. If it had been someone else no one would use it as such.

Re: Statistics with Julia [pdf]

#120

Post by mruts »

jpeloquin wrote: post #104
mruts wrote: post #61

Earlier quoted context omitted.

The classic example is getting the last element of an array. With 1-based indexing the length of the array is the index of the last element. It has a nice symmetry to it. Also I find it elegant that for 1-indexing that the start and end value for slices are both inclusive, instead of the first one being inclusive and the last being exclusive. Also, isn’t it just weird that the index of an element is one less than it’…

> Also, with 1-indexing I can multiply numbers by arrays and get reasonable offsets. 3 x 1 is three, so I would get the third element of the list. But with 0-indexing, I have 0 x 3 which gives me the same element, clearly inconsistent. This is interesting. Suppose the task is to use this approach (index * stride) to pick every third item from a list of 9 items: [1, 2, 3, 4, 5, 6, 7, 8, 9]. With 1-indexing: Multiply t…

It’s true that they both work, but I think the floating nature of the 1-indexing has some very desirable properties. If we wanted to sample from a distribution, the stride would relate to 1/n x 100 percent sampling. With 0-indexing we are always going to sample the first element. Granted this isn’t the best way to sample in the first place, so maybe it’s a contrived.
Post reply on HN