Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistics with Julia [pdf]

people.smp.uq.edu.au

Post reply on HN
Print view View original

51–60 of 136 posts

Re: Statistics with Julia [pdf]

#51

Post by Buttons840 »

j88439h84 wrote: post #32
dlphn___xyz wrote: post #26

whats the selling point with Julia? why would i use it over something like R?

It's supposed to be faster

It's a bit more nuanced than that. It's "as fast" without having to write any C.

I tried to recreate something like AlphaGo in Python using Keras, I never got the learning to work (probably because I was impatient and training on a laptop CPU), but a lot of the CPU time was simply being spent on manipulating the board state.

So I ported my "Board" object to Rust, and it was a lot faster. Things like counting liberties or removing dead stones were a lot faster, which was important.

Then I rewrote the whole thing in Julia and it was just as fast as my Python / Rust combo.

So I saw for myself that Julia does solve the two language problem. It is as pleasant to write as Python (and I like it better actually), and performed as well as Rust, based on my informal benchmarks.

Re: Statistics with Julia [pdf]

#52

Post by j88439h84 »

mruts wrote: post #42
j88439h84 wrote: post #35

Earlier quoted context omitted.

Does it have a type checker like mypy? Python's protocols ( https://www.python.org/dev/peps/pep-0544/ ) are "structured enough for large programs".

Julia has static typing. I wouldn’t go that far and say Python is suitable for large programs. It’s clearly not. Working on a large python code base is hell.

Old Python, before mypy, attrs/dataclasses, etc., is a pain. Nowadays with modern tooling, it's terrific.

Re: Statistics with Julia [pdf]

#53

Post by j88439h84 »

Buttons840 wrote: post #51
j88439h84 wrote: post #32

Earlier quoted context omitted.

It's supposed to be faster

It's a bit more nuanced than that. It's "as fast" without having to write any C. I tried to recreate something like AlphaGo in Python using Keras, I never got the learning to work (probably because I was impatient and training on a laptop CPU), but a lot of the CPU time was simply being spent on manipulating the board state. So I ported my "Board" object to Rust, and it was a lot faster. Things like counting libertie…

What's the nuance? It's much faster?

Re: Statistics with Julia [pdf]

#54

Post by kbd »

mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

I can't believe I'm jumping into the inevitable 1-based indexing discussion, but I'm surprised to see you say that one-based indexing results in "less "+ 1" or "- 1" things in your code". Most arguments I've seen come out to "it's fine" (certainly) or "it's more comfortable for mathematicians" (which I can't speak to).

Besides Dijkstra's classic paper[1] showing why 0-based indexing is superior, in practice I find myself grateful for 0-based indexing in Python because of how slices and things just work out without needing +1/-1.

I'd like to understand. Could you give an example of when 1-based indexing works out better than 0-based?

[1] http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

Re: Statistics with Julia [pdf]

#55

Post by kgwgk »

mruts wrote: post #40
kgwgk wrote: post #27

Earlier quoted context omitted.

> Julia is everything python could have been The goals of Python were quite different from the goals of Julia.

I’m not sure what Python’s goals are to be honest. It seems to me that the language is outclassed in every way by better, more consistent, more powerful, and more performant languages. Python programmers seem content implementing the same things over and over again. Like, for example, flattening a list/monad. List of things python doesn’t have but should: pattern matching, multi-line lambdas, more data structures (lo…

"In a 1999 report, Van Rossum highlighted the following as his goals for Python:

It should be an easy and intuitive language, just as powerful as major competitors.

It should be open source, so anyone can contribute to its development.

Its code should be understandable as plain English.

It should be suitable for everyday tasks, allowing for short development times."

https://www.computerhistory.org/fellowawards/hall/guido-van-...

“The first sound bite I had for Python was, "Bridge the gap between the shell and C."

So I never intended Python to be the primary language for programmers, although it has become the primary language for many Python users. It was intended to be a second language for people who were already experienced programmers, as some of the early design choices reflect.‘

https://www.artima.com/intv/pyscaleP.html

Re: Statistics with Julia [pdf]

#56

Post by j88439h84 »

mruts wrote: post #40
kgwgk wrote: post #27

Earlier quoted context omitted.

> Julia is everything python could have been The goals of Python were quite different from the goals of Julia.

I’m not sure what Python’s goals are to be honest. It seems to me that the language is outclassed in every way by better, more consistent, more powerful, and more performant languages. Python programmers seem content implementing the same things over and over again. Like, for example, flattening a list/monad. List of things python doesn’t have but should: pattern matching, multi-line lambdas, more data structures (lo…

Interesting list, thanks. FWIW, my view on these..

- I agree about pattern matching, that'd be nice.

- Multi-line lambdas haven't been important, but maybe I'm missing something.

- The list of data structures in the stdlib doesn't matter to me, since the 200k libraries on PyPI make up for it, and since packaging is easy nowadays with Poetry, they are as good as built-in but they get more frequent fixes and improvements than would be possible for the stdlib. Maybe there are some good side-effects of having extra types built in, based on a community of people using these types?

- Threading, I suppose, though Python isn't really the right language overall for that stuff anyway.

- Options, and monads, yeah that'd be nice.

- Futures are an idea whose time has come and gone IMO, but they're in asyncio anyway :\.

- For performance, PyPy is quite fast for many use cases.

Is there a language that has all these things built in and has a repl?

Re: Statistics with Julia [pdf]

#57

Post by cshenton »

j88439h84 wrote: post #53
Buttons840 wrote: post #51

Earlier quoted context omitted.

It's a bit more nuanced than that. It's "as fast" without having to write any C. I tried to recreate something like AlphaGo in Python using Keras, I never got the learning to work (probably because I was impatient and training on a laptop CPU), but a lot of the CPU time was simply being spent on manipulating the board state. So I ported my "Board" object to Rust, and it was a lot faster. Things like counting libertie…

What's the nuance? It's much faster?

Julia code gets compiled native via LLVM so it is about as fast as other natively compiled languages.

Re: Statistics with Julia [pdf]

#58

Post by cshenton »

mruts wrote: post #14

Julia is everything python could have been, and much more. I'm stuck with python right now as a lot of people in the data science/ML community are, but it's becoming increasingly viable to use Julia for "real" work. The Python-Julia interop story is pretty strong as well, which allows you to (somewhat) easily convert pandas/pytorch/sklearn code into Julia using Python wrappers. Julia has some unconventional things in…

One reason I really like the 1 based indexing is that I can have a UInt index and 0 can act as a sentinel value. Really nice for writing things like vector embedded linked lists.

Re: Statistics with Julia [pdf]

#59

Post by pjmlp »

orbifold wrote: post #37
aurelian15 wrote: post #16

Earlier quoted context omitted.

I just looked into Julia (1.1) for scientific use (simulation of very simple dynamical systems) a few days ago. I have to admit that by the end of the day I was surprisingly frustrated. I felt that type annotations were insufficient (one of the reasons to move away from Python); in particular, I didn't find a way to specify statically sized array types as you can do with Eigen, a feature that I find incredibly useful…

Julia is what happens if you let amateurs develop a compiler. The few times I’ve tried it produced gigabytes worth of stuff super slowly. The majority of packages are half backed, the only way to discover any type error is to let the program run, which coupled with multi method dispatch and hellishly slow compile times for trivial amounts of code makes the whole experience super unpleasant. Modern C++ plus some pytho…

In case you missed, LLVM and GCC enjoy lots of contributions from amateurs.

Maybe they should stop accepting them then.

Re: Statistics with Julia [pdf]

#60

Post by j88439h84 »

mruts wrote: post #42
j88439h84 wrote: post #35

Earlier quoted context omitted.

Does it have a type checker like mypy? Python's protocols ( https://www.python.org/dev/peps/pep-0544/ ) are "structured enough for large programs".

Julia has static typing. I wouldn’t go that far and say Python is suitable for large programs. It’s clearly not. Working on a large python code base is hell.

> Julia has static typing.

Julia has its own type system, which doesn't conform to the traditional static/dynamic divide. But AFAIK it doesn't have a compile-time type checker like mypy to help me catch type errors early.

Post reply on HN