Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

101–110 of 136 posts

Re: Statistics with Julia [pdf]

#101

Post by Jach »

j88439h84 wrote: post #52
mruts wrote: post #42

Earlier quoted context omitted.

Julia has static typing. I wouldn’t go that far and say Python is suitable for large programs. It’s clearly not. Working on a large python code base is hell.

Old Python, before mypy, attrs/dataclasses, etc., is a pain. Nowadays with modern tooling, it's terrific.

How much do you leverage the REPL when you're developing? In my experience with "old Python" (I've yet to update my resume with new Python) I developed in an incremental manner making heavy use of the REPL, I never had the pains I do in static languages without a REPL story. Your comment to me reads like "Nowadays with modern tooling [that lets me develop Python like Java developers develop Java], it's terrific." My day job is Java with a powerful IDE that is crucial for my productivity, but this is because I'm trapped in the "write a comet mass of code interacting with a solar mass of other code and lean on the compiler and IDE tools in addition to my reason to tell me it has a chance of working" way of developing rather than sending small chunks of code to a REPL, testing immediately, and building things up. If I had to (or only knew how to) develop old Python in the Java way I'd probably go mad too. TDD can almost sometimes substitute for a REPL in JavaLand, but it has its own flaws... (And yes modern Java now has something like a REPL, my work environment unfortunately hasn't updated to support it. Soon.)

Re: Statistics with Julia [pdf]

#102

Post by j88439h84 »

j88439h84 wrote: post #34
jointpdf wrote: post #20

This looks like a good reference for the fundamentals of both statistics and Julia, as claimed. I have a small critique, since the authors asked for suggestions. The format for the code samples goes like (code chunk —> output/plots —> bullet points explaining the code line-by-line). This creates a bit of a readability issue. The reader will likely follow a pattern like: (Skim past the code chunk to the explanation —>…

I'm not sure how that allennlp site is doing it, but source is here: https://github.com/allenai/allennlp/blob/b0ea7ab6be2787495fa...

Here's what they're doing: https://github.com/allenai/allennlp/blob/master/tutorials/ta...

Re: Statistics with Julia [pdf]

#103

Post by j88439h84 »

Jach wrote: post #101
j88439h84 wrote: post #52

Earlier quoted context omitted.

Old Python, before mypy, attrs/dataclasses, etc., is a pain. Nowadays with modern tooling, it's terrific.

How much do you leverage the REPL when you're developing? In my experience with "old Python" (I've yet to update my resume with new Python) I developed in an incremental manner making heavy use of the REPL, I never had the pains I do in static languages without a REPL story. Your comment to me reads like "Nowadays with modern tooling [that lets me develop Python like Java developers develop Java], it's terrific." My…

I use the repl plenty. I can write the types after I figure out what I'm writing, or before, if I want them to guide me.

Re: Statistics with Julia [pdf]

#104

Post by jpeloquin »

mruts wrote: post #61
kbd wrote: post #54

Earlier quoted context omitted.

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

The classic example is getting the last element of an array. With 1-based indexing the length of the array is the index of the last element. It has a nice symmetry to it. Also I find it elegant that for 1-indexing that the start and end value for slices are both inclusive, instead of the first one being inclusive and the last being exclusive. Also, isn’t it just weird that the index of an element is one less than it’…

> Also, with 1-indexing I can multiply numbers by arrays and get reasonable offsets. 3 x 1 is three, so I would get the third element of the list. But with 0-indexing, I have 0 x 3 which gives me the same element, clearly inconsistent.

This is interesting. Suppose the task is to use this approach (index * stride) to pick every third item from a list of 9 items: [1, 2, 3, 4, 5, 6, 7, 8, 9].

With 1-indexing: Multiply the sequence of valid indices (1, 2, 3, ...) by the stride (3) and use the result to 1-index into the given list. Returns [3, 6, 9].

With 0-indexing: Multiply the sequence of valid indices (0, 1, 2, ...) by the stride (3) and use the result to 0-index into the given list. Returns [1, 4, 7].

0-indexing has the start point of the return values fixed to the origin. 1-indexing has its start point float around depending on the stride. Both work, but have different emergent properties in the given example.

Re: Statistics with Julia [pdf]

#105

Post by Jach »

j88439h84 wrote: post #103
Jach wrote: post #101

Earlier quoted context omitted.

How much do you leverage the REPL when you're developing? In my experience with "old Python" (I've yet to update my resume with new Python) I developed in an incremental manner making heavy use of the REPL, I never had the pains I do in static languages without a REPL story. Your comment to me reads like "Nowadays with modern tooling [that lets me develop Python like Java developers develop Java], it's terrific." My…

I use the repl plenty. I can write the types after I figure out what I'm writing, or before, if I want them to guide me.

Sounds closer to a lisp workflow then, excellent! Thanks for replying as it helps me update away from the notion that it's typically the development style that leads to pains in dynamic languages more than the dynamic/static divide itself and people's preferences.

Re: Statistics with Julia [pdf]

#106

Post by ddragon »

abakus wrote: post #100

I find Julia's .> , .==, .*, ./ (dots for element-by-element ufunc)... really ugly. Numpy's design is cleaner and better.

Why? When I see the '.' I immediately know it's a broadcasted function (for example * for matrix multiplication vs *. hadamard product), and I get the vectorized version of any function I write for free with no extra boilerplate (and the compiler will even automatically fuse them together if I chain them to avoid wasting allocations). You can even customize the broadcasting and the fusion.

Re: Statistics with Julia [pdf]

#107

Post by eigenspace »

kazinator wrote: post #99
eigenspace wrote: post #90

Earlier quoted context omitted.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

Zero based indexing objectively has various convenient properties which one-based indexing doesn't. The value of convenience over inconvenience isn't objectively better, that's all. Objectively speaking, if I find the least positive residue of some integer modulo M, I get a value from 0 to M-1. If my M-sized array is from 0 to M-1, that is objectively convenient: hash_table[hash(string) % table_size] Objectively spea…

You do realize that there are approximately just as many use cases where 1 based indexing is more natural and involves less operations, right? It’s highly problem and context dependant.

But I’m not trying it make the point that 0 based or 1 based is better or worse than the other. I’m just saying that it’s a borderline immaterial difference for most use-cases and Julia gives many many tools for getting around any problems that may arise when a certain indexing scheme is awkward.

The 0 or 1 based debate is one of the most boring and pedantic arguments one can have and I do my best to ridicule people when they try to start it.

Re: Statistics with Julia [pdf]

#108

Post by jebej »

anonova wrote: post #18
caiocaiocaio wrote: post #6

Julia looked interesting to me, so I tried 1.0 after it came out. I have a oldish laptop (fine for my needs), and every time I tried to do seemingly anything, it spent ~5 minutes recompiling libraries or something. So I've been waiting newer versions that hopefully stop doing that, or for me to buy a better computer.

Yes, this is ones of my problems with Julia. It seems to be optimized for long runs and REPL/notebook usage. Take, for example, a simple program that creates a line plot ( https://docs.juliaplots.org/latest/tutorial/ ): using Plots x = 1:10 y = rand(10) plot(x, y) After installing the package, the first run has to precompile(?), and subsequent runs use the package cache. But ~25 s to create a simple plot is incredibl…

The Plots package adds a significant overhead right now. Try using PyPlot (matplotlib) directly. These days you can use exactly the same syntax (dot-call) as in Python.

   $ time julia -e "using PyPlot;x=1:10;y=rand(10);plot(x,y);"
   real    0m5.676s

Re: Statistics with Julia [pdf]

#109

Post by jjoonathan »

tomrod wrote: post #95
jjoonathan wrote: post #89

Earlier quoted context omitted.

I don't just think it's a feature, I think it's a killer feature. You are much less likely to reinvent the wheel if you can add your one critical niche feature / bugfix to an existing library. In python, learning C and C build systems and python's C API are gigantic barriers to doing that. More importantly, if every fast data manipulation needs to be written in C, a few of them can be profitably shared, but you need…

Maybe I don't understand what API bloat is in this context -- can you give some more detail regarding your thoughts on pandas?

Here's one of the fifteen API ref sections in pandas:

https://pandas.pydata.org/pandas-docs/stable/reference/serie...

Even though it's long, it undersells the problem, because many of these methods have nontrivial overload semantics that open up like a fractal when you look in turn at their docs. The link also undersells the problem because this junkheap is evidently so incomplete that people are frequently forced to rely on numpy to extend it.

APIs should make hard things easy, but API gloveboxes like this make easy things hard. Minimal API + Performant Glue >> We do everything for you + You can't ever touch your own data or your perf dies + Good luck reverse engineering these semantics if you've forgotten the context and need to port it.

Re: Statistics with Julia [pdf]

#110

Post by newen »

ViralBShah wrote: post #86
3jckd wrote: post #81

Earlier quoted context omitted.

Yes, they are. Slow and hardly as expressive or rich as python/r counterparts.

One can use matplotlib in Julia by PyCall'ing it. So it is at least as good as anything else.

Or ggplot2 using RCall, which is what I use and it's quite nice.
Post reply on HN