Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

91–100 of 136 posts

Re: Statistics with Julia [pdf]

#91

Post by kbd »

eigenspace wrote: post #90
iamed2 wrote: post #83

Earlier quoted context omitted.

Dijkstra's write-up is full of subjective aesthetic judgements that certain things are ugly. I personally don't find `1:0` for an empty sequence to be ugly, and I do find using `1:1` to refer to an empty sequence and `1:2` to refer to the sequence `{1}` to be ugly. I would encourage everyone to read over his reasoning and see if you agree with his aesthetic judgements.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

I wasn't arguing so much that it was "objective proof", but that in contrast to "The reason for zero indexing is historical, related to pointer offsets. I don’t think anyone chose them to be easier for people.", I was pointing out that there are arguments for it as well. It's not just because "the address of an array is also the address of its first element in C".

Re: Statistics with Julia [pdf]

#92

Post by eigenspace »

kbd wrote: post #54
mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

Shouldn’t Dijkstra’s paper be your 0th reference?

Re: Statistics with Julia [pdf]

#93

Post by stabbles »

kbd wrote: post #54
mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

In C++ you typically access arrays with unsigned integers (size_t), and a common pitfall is:

    for (size_t i = v.size() - 1; i >= 0; --i) {
      std::cout
To fix the infinite loop you could write:

    for (size_t i = v.size(); i > 0; --i) {
      std::cout
Neither is great. Switching to signed integers might make your compiler throw warnings at you.

However, 1-based indexing does not work out well with modular arithmetic:

    # 1 based
    v[1 + (i - 1) % v.size()]

    # 0 based
    v[i % v.size()]
There's pros and cons with both schemes.

Re: Statistics with Julia [pdf]

#94

Post by ddragon »

stabbles wrote: post #93
kbd wrote: post #54

Earlier quoted context omitted.

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to). Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find my…

In C++ you typically access arrays with unsigned integers (size_t), and a common pitfall is: for (size_t i = v.size() - 1; i >= 0; --i) { std::cout To fix the infinite loop you could write: for (size_t i = v.size(); i > 0; --i) { std::cout Neither is great. Switching to signed integers might make your compiler throw warnings at you. However, 1-based indexing does not work out well with modular arithmetic: # 1 based v…

Yes, and in both cases the language should provide tools so you don't have to deal directly with those edge cases. For example in Julia, for modular arithmetic with 1-based indexing there is mod1 [1], and for iterating in Julia you should use eachindex which will always work for both 0 or 1 indexed arrays.

[1] https://docs.julialang.org/en/v1/base/math/#Base.mod1

[2] https://docs.julialang.org/en/v1/base/arrays/index.html#Base...

Re: Statistics with Julia [pdf]

#95

Post by tomrod »

jjoonathan wrote: post #89
opportune wrote: post #85

Earlier quoted context omitted.

>you can typically just write down the code you want to write, rather than being forced to find a library that wraps a C/C++ implementation like in python/r. I don't think this is really a feature. It's nice that you can write more performant code in Julia directly and don't need to wrap lower level languages, without question, but the lack of libraries or library features is not a good thing. It's always better to u…

I don't just think it's a feature, I think it's a killer feature. You are much less likely to reinvent the wheel if you can add your one critical niche feature / bugfix to an existing library. In python, learning C and C build systems and python's C API are gigantic barriers to doing that. More importantly, if every fast data manipulation needs to be written in C, a few of them can be profitably shared, but you need…

Maybe I don't understand what API bloat is in this context -- can you give some more detail regarding your thoughts on pandas?

Re: Statistics with Julia [pdf]

#96

Post by tomrod »

ViralBShah wrote: post #87
anonova wrote: post #18

Earlier quoted context omitted.

Yes, this is ones of my problems with Julia. It seems to be optimized for long runs and REPL/notebook usage. Take, for example, a simple program that creates a line plot ( https://docs.juliaplots.org/latest/tutorial/ ): using Plots x = 1:10 y = rand(10) plot(x, y) After installing the package, the first run has to precompile(?), and subsequent runs use the package cache. But ~25 s to create a simple plot is incredibl…

The time to second plot will be a few milliseconds, in the same process - in the same Julia session. So, while the time to first plot is frustrating, it is ok if your interactive session times are longer. Of course, we continue to work on improving compile times. About half of the time is spent in LLVM compilation, which has actually become slower over time.

What prevents the plot compilation from being pre-compiled at install?

Re: Statistics with Julia [pdf]

#97

Post by dmix »

pjmlp wrote: post #59
orbifold wrote: post #37

Earlier quoted context omitted.

Julia is what happens if you let amateurs develop a compiler. The few times I’ve tried it produced gigabytes worth of stuff super slowly. The majority of packages are half backed, the only way to discover any type error is to let the program run, which coupled with multi method dispatch and hellishly slow compile times for trivial amounts of code makes the whole experience super unpleasant. Modern C++ plus some pytho…

In case you missed, LLVM and GCC enjoy lots of contributions from amateurs. Maybe they should stop accepting them then.

Amateurs developing a compiler != amateurs contributing to a compiler. Every popular project has clear project leads who guide most important decision making.

Not that I have any position on Julia

Re: Statistics with Julia [pdf]

#98

Post by ChrisRackauckas »

opportune wrote: post #85
superdimwit wrote: post #38

I'd really recommend anyone doing mildly numerical / data-ey work in python to give Julia a patient and fair try. I think the language is really solidly designed, and gives you ridiculously more power AND productivity than python for a whole range of workloads. There are of course issues, but even in the short time I've been following & using the language these are being rapidly addressed. In particular: generally le…

>you can typically just write down the code you want to write, rather than being forced to find a library that wraps a C/C++ implementation like in python/r. I don't think this is really a feature. It's nice that you can write more performant code in Julia directly and don't need to wrap lower level languages, without question, but the lack of libraries or library features is not a good thing. It's always better to u…

While Python has good libraries in general computing, and it has good ML libraries, it's really lacking in scientific computing (numerical linear algebra, differential equations, etc.). For example, what's a Newton-Krylov IMEX integrator in Python? Boundary value DAEs? I know of libraries for these things in Fortran, C++, and Julia... but not Python. It's also well-known that Python lacks a lot of the statistics libraries of R. When you chart it out, Python tends to just have the bare minimum of support in every area (except ML, it has good ML libraries), which if it's what you need, great! But...

Re: Statistics with Julia [pdf]

#99

Post by kazinator »

eigenspace wrote: post #90
iamed2 wrote: post #83

Earlier quoted context omitted.

Dijkstra's write-up is full of subjective aesthetic judgements that certain things are ugly. I personally don't find `1:0` for an empty sequence to be ugly, and I do find using `1:1` to refer to an empty sequence and `1:2` to refer to the sequence `{1}` to be ugly. I would encourage everyone to read over his reasoning and see if you agree with his aesthetic judgements.

I’m constantly baffled by the way people hold that paper up as some sort of objective proof that 0 based indexing is superior.

Zero based indexing objectively has various convenient properties which one-based indexing doesn't.

The value of convenience over inconvenience isn't objectively better, that's all.

Objectively speaking, if I find the least positive residue of some integer modulo M, I get a value from 0 to M-1. If my M-sized array is from 0 to M-1, that is objectively convenient:

  hash_table[hash(string) % table_size]
Objectively speaking, if I have some files in a directory and I give them zero based names like file000, file001, ... then I can objectively refer to the first volume of ten using the single pattern file00?, and the next volume as file01?. If they are numbered from 001, I need file00? file010 to match the first ten. For the next ten, the file01? pattern unfortunately matches file010 so I objectively need some way to exclude it.

Objectively speaking, if I have zero based indices to a 3D array, I can find the address of an element using the homogeneous Ai + Bj + Ck rather than Ai + Bj + Ck + D, which objectively adds an extra term.

Objectively speaking, the zero-based byte index B can be converted to a four byte word index using B / 4 (truncating division), and within that word, the zero-based byte local offset is B % 4. Objectively speaking, the same conversion from 1-based bytes to 1-based words requires (B-1)/4+1 and (B-1)%4+1, which is objectively more syntax and more nodes in the abstract syntax tree.

There is no reason I should like shorter, simpler, faster; that's a purely subjective aesthetic. After all, a short poem isn't better than a long one; a bacterium isn't better than a rhinoceros; and so on.

Hey, how about those one-based music intervals? C to E is a major third and all that? We have a diatonic scale with seven notes, right? As we ascend, whenever we cycle through seven notes, we have passed ... one more octave. And to invert an interval, we subtract from ... why, nine of course! And a fifth stacked on top of a fifth is a major ninth. There is objectively more cruft in 1 based music theory than 0 based. But there is no accounting for people liking it that way, right?

Post reply on HN