Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R, the master troll of statistical languages (2012)

talyarkoni.org

Post reply on HN
Print view View original

71–80 of 154 posts

Re: R, the master troll of statistical languages (2012)

#71

Post by Cogito »

GFK_of_xmaspast wrote: post #69
existencebox wrote: post #33

Earlier quoted context omitted.

It's funny you bring up python; I say this not as a comment on your thesis, but related, since I often hear the "python is slow" trope but that's only half true, you can typically write python that is plenty "fast enough" (As a day-jobbing data pipeline engineer) if you're implementing with an understanding of what things will drive you into the mud. This goes beyond just understanding the tool you're using, fundamen…

I've translated plenty of numerical code from (pure-ish) python to c and c++, and usually get about a 100x speedup, sometimes as high as 800x, implementing the same algorithms.

Any idea where the speedups came from? Is it that the problems weren't algorithmically limited in the first place (lots of io for example), reduction of overhead etc (what kind of python was the code running on before?), or just that the speedup on low level operations added up cumulatively and cam e to dominate the other timing factors?

Also, did you change the data structures or use the same ones as in python? Was any of the speed boost data structure related?

Re: R, the master troll of statistical languages (2012)

#72

Post by makeset »

JasonCEC wrote: post #19
evandev wrote: post #14

My thoughts (a little of a Rant) on R as the Lead Engineer at a data science focused company is that R is a great statistical language, but a poor programming language. I use the term programming language as a language which is very versatile for a variety of needs (web app, commandline app) such as python, ruby, etc. R has the capabilities to use as a programming language like a climbing rope can be used as a belt.…

To contrast this - I'm the lead data scientist at the same company, and head over heels in love with R.... It is the only language I can quickly and efficiently jump from algebraic topology for novel pre-processing, straight into model building and validation - with just about every potential variation of every major algorithm freely available and packaged on a well curated package manager (CRAN), and then ensemble t…

R's server sockets can't select(), so unless you've reimplemented that as well, don't count on handling concurrent requests with that web server.

Re: R, the master troll of statistical languages (2012)

#73

Post by existencebox »

GFK_of_xmaspast wrote: post #69
existencebox wrote: post #33

Earlier quoted context omitted.

It's funny you bring up python; I say this not as a comment on your thesis, but related, since I often hear the "python is slow" trope but that's only half true, you can typically write python that is plenty "fast enough" (As a day-jobbing data pipeline engineer) if you're implementing with an understanding of what things will drive you into the mud. This goes beyond just understanding the tool you're using, fundamen…

I've translated plenty of numerical code from (pure-ish) python to c and c++, and usually get about a 100x speedup, sometimes as high as 800x, implementing the same algorithms.

At the risk of being overly pragmatic, note that I said "fast enough" and not "as fast as possible."

My comment was more on the perception that python is unworkably slow in many situations, where I can count the number of times on my hands that I've NEEDED to C-ify some hot paths.

If you're writing a plasma fluid simulation to run on a HHPC cluster, yes, you probably damn well want some straight C/C++. Outside of similarly exceedingly high throughput situations, CPUs are normally more than fast enough, especially if the application in any way brushes up against people and thus falls into "human time" scales, in which case you'd typically be hard pressed to make things slow enough for someone to notice. (Yet somehow we find a way...)

To a sister post re: where python->C speedup can occur, to two birds with one stone, I imagine there's a lot of low hanging fruit, to take one obvious one, anything the compiler can optimize away. Memory read/address optimization, vectorization, potentially better support for branch prediction, I can handwave at more but I am so far from a compilers type that I'd probably make a fool of myself.

Re: R, the master troll of statistical languages (2012)

#74

Post by klmr »

Cogito wrote: post #71
GFK_of_xmaspast wrote: post #69

Earlier quoted context omitted.

I've translated plenty of numerical code from (pure-ish) python to c and c++, and usually get about a 100x speedup, sometimes as high as 800x, implementing the same algorithms.

Any idea where the speedups came from? Is it that the problems weren't algorithmically limited in the first place (lots of io for example), reduction of overhead etc (what kind of python was the code running on before?), or just that the speedup on low level operations added up cumulatively and cam e to dominate the other timing factors? Also, did you change the data structures or use the same ones as in python? Was…

Python and similar dynamic languages suffer from the fact that every name access (variable, function, etc) incurs a dynamic lookup of that name in a (nested) dictionary. Statically compiled languages don’t have this. There are fairly recent, clever optimisations that can avoid many of these lookups but (a) they are not implemented in any of the common implementations of Python, R, etc (JavaScript has them though). But even with these optimisations in place we cannot get rid of such lookups altogether, and they kill cache locality and branch prediction.

There are other reasons for slowdown (automatically managed garbage collection is a big one, and so is any kind of indirection, e.g. callbacks). But usually the big one is name lookup.

Re: R, the master troll of statistical languages (2012)

#75

Post by klmr »

gbrown wrote: post #45
chrisseaton wrote: post #41

Earlier quoted context omitted.

Why can't a JIT solve this? It shouldn't need to do the bookkeeping for every iteration if it has JIT compiled it. A JIT should be able to take advantage of processor vector instructions etc.

R does actually ship with the ability to byte-compile functions these days, and as that functionality matures it may become the default behavior. It's still better to actually learn the language; it's far easier to optimize something like: apply(X, 1, function(x){ # do stuff to the row of X }) than: for (i in 1:nrow(X)){ # do stuff to X[i,], and store it somewhere }

As far as I know byte-compiling won’t actually alleviate the repeated name lookup (or does it?). Unless the R byte compiler is fiendishly clever, every single name lookup in the loop body will still incur what essentially amounts to a `get(name, environment(), inherits = TRUE)` call.

Re: R, the master troll of statistical languages (2012)

#77

Post by mziel »

DangerousPie wrote: post #63

I have my fair share of problems with R, but that first example (4 ways to select a column) seems a bit silly. Just off the top of my head, I could think of plenty of ways to do the same thing in Python/pandas: ice_cream.icol[0] ice_cream['col'] ice_cream.iloc[:, 0] ice_cream.loc[:, 'col'] ice_cream.ix[:, 'col'] And if you wanted to make things more convoluted, you could also wrap things into lists like the author di…

For R at least some of it is due to R's flexibility

x$name syntax stems from data frames being really lists in disguise x[["name"]] ditto, plus because it's useful to access by string (see reflection in other languages)

x[,"name"] and x[,1] because we can also apply the matrix syntax to data frames

Re: R, the master troll of statistical languages (2012)

#78

Post by klmr »

stewbrew wrote: post #10
mziel wrote: post #6

Earlier quoted context omitted.

Factor is one of the worst thing of R-world. I don't recall ever needing factors, yet they creep in with many functions (read.csv, cut). Btw there's a nice readr package (from Hadleyverse) that has a read_csv method that does away with factors by default.

How would you represent categorical data then? R's primary use case isn't text processing. And HW isn't always right.

As character, for instance (in particular, they can do everything factors can do when used in conjunction with `unique`, and sorted factors can be represented as a conjunction of characters and numerics). Factors work better, but only barely. In particular, they are nowadays not any more efficient than using character (!). They used to be, which is why they are liberally used everywhere in R’s base libraries.

Re: R, the master troll of statistical languages (2012)

#80

Post by GFK_of_xmaspast »

Cogito wrote: post #71
GFK_of_xmaspast wrote: post #69

Earlier quoted context omitted.

I've translated plenty of numerical code from (pure-ish) python to c and c++, and usually get about a 100x speedup, sometimes as high as 800x, implementing the same algorithms.

Any idea where the speedups came from? Is it that the problems weren't algorithmically limited in the first place (lots of io for example), reduction of overhead etc (what kind of python was the code running on before?), or just that the speedup on low level operations added up cumulatively and cam e to dominate the other timing factors? Also, did you change the data structures or use the same ones as in python? Was…

All from (3). Definitely not io bound, and using standard python 2.7 (if numpy had been applicable, I would have used it...)

My data structures for numerics are generally really simple, and generally I'm able to go from python list/dict/sets to c++ vector/map/set pretty directly.

Post reply on HN