Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

121–130 of 136 posts

Re: Statistics with Julia [pdf]

#121
post #95

Earlier quoted context omitted.

Maybe I don't understand what API bloat is in this context -- can you give some more detail regarding your thoughts on pandas?

Here's one of the fifteen API ref sections in pandas: https://pandas.pydata.org/pandas-docs/stable/reference/serie... Even though it's long, it undersells the problem, because many of these methods have nontrivial overload semantics that open up like a fractal when you look in turn at their docs. The link also undersells the problem because this junkheap is evidently so incomplete that people are frequently forced to…

Okay, I think I see your point. The different object methods you are seeing as API calls, and because they are granular and have capacity to do many common and uncommon tasks this is viewed as bloat. Makes sense from that perspective. Thanks.

Re: Statistics with Julia [pdf]

#122
post #101

Earlier quoted context omitted.

Old Python, before mypy, attrs/dataclasses, etc., is a pain. Nowadays with modern tooling, it's terrific.

How much do you leverage the REPL when you're developing? In my experience with "old Python" (I've yet to update my resume with new Python) I developed in an incremental manner making heavy use of the REPL, I never had the pains I do in static languages without a REPL story. Your comment to me reads like "Nowadays with modern tooling [that lets me develop Python like Java developers develop Java], it's terrific." My…

I don’t think static typing is really related to a repl. There are many statically types languages that provide great repls: Haskell, OCaml, Scala, etc.

Re: Statistics with Julia [pdf]

#123

This looks like a good reference for the fundamentals of both statistics and Julia, as claimed. I have a small critique, since the authors asked for suggestions. The format for the code samples goes like (code chunk —> output/plots —> bullet points explaining the code line-by-line). This creates a bit of a readability issue. The reader will likely follow a pattern like: (Skim past the code chunk to the explanation —>…

Thank you. Indeed not sure how to optimize it. Perhaps in the next version of the book. Note that the book is to be Springer published (once finished) - this puts some limitations as well.

Happy for more feedback (Yoni Nazarathy).

Re: Statistics with Julia [pdf]

#124

This is a very good resource. The one thing I would ask is that I would like to see examples of using DifferentialEquations.jl when you get to the section on dynamical systems, especially when doing discrete event simulation and stochastic differential equations. I opened an issue in the repo and we can continue discussing there (I'll help write the code, I want to use this in my own class :P)!

We actually use the DifferentialEquations.jl package in one of the examples: https://github.com/h-Klok/StatsWithJuliaBook/blob/master/10_...

Re: Statistics with Julia [pdf]

#125
post #66

I was going to ask is there any Kindle version of this, then I skimmed over the book, and I don't think it will be readable on a Kindle. And even if it does, the reading experience will definitely be inferior.

The book will be published by Springer (at which point the online draft will be removed).

Yoni Nazarathy.

Re: Statistics with Julia [pdf]

#126
post #54
post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

I'm not really convinced by Dijkstras paper. He is basically saying indexing from zero is more natural because if you have an array of natural numbers including zero, then the range of numbers [0..n] is denoted by the index [0..n] which is logical. With 1-indexing you have have to write [1..n+1] to get the values [0..n] which is weird and ugly. Sure, but this assumes that the array in question is starting with 0 in the first place! The whole argument is begging the question.

Re: Statistics with Julia [pdf]

#127
post #77
post #61

Earlier quoted context omitted.

The classic example is getting the last element of an array. With 1-based indexing the length of the array is the index of the last element. It has a nice symmetry to it. Also I find it elegant that for 1-indexing that the start and end value for slices are both inclusive, instead of the first one being inclusive and the last being exclusive. Also, isn’t it just weird that the index of an element is one less than it’…

> The classic example is getting the last element of an array. Good point, in Python I don't notice that the last element is arr[len(arr)-1] because Python provides arr[-1]. I think in general your point is that it's natural for the nth element to be arr[n]. > The reason for zero indexing is historical, related to pointer offsets. There is that, but Dijkstra's paper makes the case from first-principles that the close…

> Dijkstra's paper makes the case from first-principles that the closed, open interval of [0,n) for sequences is the most appropriate

He argues that it is the most appropriate when indexing into an array of the natural numbers starting with 0. If the array in question started with 1, one-based indexing would be most appropriate following exactly the same logic!

Re: Statistics with Julia [pdf]

#128

Earlier quoted context omitted.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

Zero based indexing objectively has various convenient properties which one-based indexing doesn't. The value of convenience over inconvenience isn't objectively better, that's all. Objectively speaking, if I find the least positive residue of some integer modulo M, I get a value from 0 to M-1. If my M-sized array is from 0 to M-1, that is objectively convenient: hash_table[hash(string) % table_size] Objectively spea…

"Objectively speaking" there are pros and cons to each system. The largest pro of 0-based indexing is of course that it can correspond to a memory address plus an offset, which is the reason C (and derived languages) use 0-based.

But it is also an objective fact that using 1-based indexing means that the index corresponds to the ordinal numbers, e.g. index 1 is the first element, index 2 is the second element and so on. This also have a number of convenient properties.

For example February is the 2. month, so if you have a list of the names of the months, you would expect month_names[2] to be February. With zero-based you would have to do month_names[month_number - 1]. And if you want to get the month number from the name, you would have to do month_names.index_of(month_name) + 1. Be careful not to switch up the +1 and -1!

As for music theory, a third is called so because it spans three half-notes. It describe the size of a range which is independent of the offset of the indices. By the same token decades are 10 years (not 9) and centuries are 100 years (not 99).

Re: Statistics with Julia [pdf]

#129
post #83

Earlier quoted context omitted.

Dijkstra's write-up is full of subjective aesthetic judgements that certain things are ugly. I personally don't find `1:0` for an empty sequence to be ugly, and I do find using `1:1` to refer to an empty sequence and `1:2` to refer to the sequence `{1}` to be ugly. I would encourage everyone to read over his reasoning and see if you agree with his aesthetic judgements.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

I think people get tricked by his way of building an argument. He structures it rhetorically almost like it is a mathematical proof, but if you read carefully the central argument is that it is "nicer"...in the particular example he picked. But if you don't pay attention it might feel like he "proves" that 0-based is universally better.

Then he goes on weird diatribes like the one about "corporate religion". I'm not sure, but I think he is trying to say that people who question his dubious logic are "sheeple".

Re: Statistics with Julia [pdf]

#130
post #128

Earlier quoted context omitted.

Zero based indexing objectively has various convenient properties which one-based indexing doesn't. The value of convenience over inconvenience isn't objectively better, that's all. Objectively speaking, if I find the least positive residue of some integer modulo M, I get a value from 0 to M-1. If my M-sized array is from 0 to M-1, that is objectively convenient: hash_table[hash(string) % table_size] Objectively spea…

"Objectively speaking" there are pros and cons to each system. The largest pro of 0-based indexing is of course that it can correspond to a memory address plus an offset, which is the reason C (and derived languages) use 0-based. But it is also an objective fact that using 1-based indexing means that the index corresponds to the ordinal numbers, e.g. index 1 is the first element, index 2 is the second element and so…

Some machine-level implementation convenience is the smallest advantage. Zero based would be better even if it cost more at the machine level. Of course it doesn't cost more because the advantages are relevant at the implementation level also.

> For example February is the 2. month, so if you have a list of the names of the months, you would expect month_names[2] to be February.

That's not 1-based indexing being good; that's conforming to (or reflecting) an externally imposed 1-based system that is itself questionable.

Should the seconds of a minute go from 1 to 60 instead of 0 to 59? Dates and times are full of poorly chosen conventions, including ones that don't match people's intuitions. For instance, many people celebrated the new millennium in January 2000. People also want decades to go from 0 to 9; the "eighties" are years matching the pattern 198?, not 1981 to 1990. Yet the 20th century goes from 1901 to 2000.

In many situations when 1 based numbering is used, it's just symbols in a sequence. It could be replaced by Gray code, greek letters, or Japanese kana in i-ro-ha order.

When the arithmetic properties of the index matter to the point that it's involved in multiplication (not merely successor/predecessor relationship), it is advantageous to make the origin zero.

If month_name[1] must be "January", I'm okay with wasting month_name[0]; that's better than supporting a one-based array (let alone making that default).

> As for music theory, a third is called so because it spans three half-notes.

No it doesn't; in that very same music theory, a "major second" interval is also known as "one step" or a "whole step"! A third is "two steps"; that's what it spans. (I don't know what you mean by "half-notes"; I sense confusion.) This nonsense was devised centuries ago by innumerates, just like various aspects of the calendar.

Post reply on HN