Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R, the master troll of statistical languages (2012)

talyarkoni.org

Post reply on HN
Print view View original

51–60 of 154 posts

Re: R, the master troll of statistical languages (2012)

#51

Post by gbrown »

mziel wrote: post #6
louden wrote: post #3

R is a language with a lot of gotcha's. I usually get burned by characters being converted to factors in read.csv() and converting factors to numeric (it works, but not how you intend). The R Inferno ( http://www.burns-stat.com/documents/books/the-r-inferno/ ) has a lot of other gotcha's and is worth a read for people who use the language. That said, the power, flexibility and user community make it my go-to for any…

Factor is one of the worst thing of R-world. I don't recall ever needing factors, yet they creep in with many functions (read.csv, cut). Btw there's a nice readr package (from Hadleyverse) that has a read_csv method that does away with factors by default.

Factors are great, and surprisingly powerful even outside of statistical computing. With that being said, I prefer to create them on purpose rather than having read.csv attempting to be helpful.

Re: R, the master troll of statistical languages (2012)

#52

Post by x0x0 »

gbrown wrote: post #49
x0x0 wrote: post #44

Earlier quoted context omitted.

There's some movement in that direction. However, the R core committers are essentially not only volunteers, but they're all (afaik) academic statisticians. One of the people who made strides in this direction is primarily an computational statistician at Iowa (Luke Tierney / compiler package). Building a high performance runtime/jit is wildly out of their scope of expertise. In retrospect, and I think many of them w…

I don't think calling Luke an "agricultural statistician" is at all reflective of his work. Not everything in Iowa is corn, and Luke has been working in computationally intensive statistical methodology and statistical software development for decades.

Hmm, you're quite right; I'm not sure how I came to believe that.

Re: R, the master troll of statistical languages (2012)

#53

Post by bachmeier »

chrisseaton wrote: post #25
mziel wrote: post #4

The problem is people using R without trying to learn about the language itself, just assuming it works like their favourite language. For example complaining that R is slow and then writing iterative solution instead of using vectorization. When I saw the example the author gave my first thought was "sapply/lapply". Lapply is essential to the R use, and is being taught early on in every book/course on R I've ever sa…

> For example complaining that R is slow and then writing iterative solution instead of using vectorisation. But if someone prefers iterative solutions, or that's all they know, why can't R make them just as fast as the vectorised versions?

Keep in mind just how old the language is, as it started as S in 1976. It was intended to be a glue language for Fortran and C.

Keep in mind also that it's easy to rewrite the bottlenecks (which are only a small part of most programs) in C, C++, Fortran and other languages including D. That may not be what any particular person is looking for, but that's traditionally the way things have been done.

Re: R, the master troll of statistical languages (2012)

#55

Post by bachmeier »

superuser2 wrote: post #5

Most of my university classmates' first exposure to programming is using R in a statistics class. It's awful. I wish they'd make Python or something a prerequisite, so that giant swaths of people don't get turned off of computing or start with the strange ideas it teaches.

It sounds like you are arguing for imperative programming over functional programming.

Re: R, the master troll of statistical languages (2012)

#56

Post by sooheon »

nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

I wonder what disagreements or counterarguments downvoters might have, and if they might share them.

For the parent, given what you've observed, do you still go to R for data crunching, or have you found anything in Clojure land that measures up?

Re: R, the master troll of statistical languages (2012)

#57

Post by jonchang »

nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

See, this is another interesting example of the kind of behavior described here: https://news.ycombinator.com/item?id=11113042

People who don't take the time to learn the language are having to go through these contortions to make R work the way their favorite language works, rather than just taking the time to learn how R works!

    R has a useful function, paste, that concatenates strings together.
    Only it takes varargs, not a character vector, so if you have a
    vector v of strings, you have to use do.call(paste, v).
But the help for the `paste` function literally goes over this exact situation:

    > v  paste(v, collapse = ",")
    [1] "1,2,3,4,5"
I'm often super baffled by the lengths people will go to not figure out how to use R and insist on writing language in R.

Re: R, the master troll of statistical languages (2012)

#58

Post by kgwgk »

gbrown wrote: post #49
x0x0 wrote: post #44

Earlier quoted context omitted.

There's some movement in that direction. However, the R core committers are essentially not only volunteers, but they're all (afaik) academic statisticians. One of the people who made strides in this direction is primarily an computational statistician at Iowa (Luke Tierney / compiler package). Building a high performance runtime/jit is wildly out of their scope of expertise. In retrospect, and I think many of them w…

I don't think calling Luke an "agricultural statistician" is at all reflective of his work. Not everything in Iowa is corn, and Luke has been working in computationally intensive statistical methodology and statistical software development for decades.

He created lisp-stat in the late 80's

https://www.jstatsoft.org/article/view/v013i09

"While R and Lisp are internally very similar, in places where they differ the design choices of Lisp are in many cases superior. The difficulty of predicting performance and hence writing code that is guaranteed to be efficient in problems with larger data sets is an issue that R will need to come to grips with, and it is not likely that this can happen without some significant design changes."

Re: R, the master troll of statistical languages (2012)

#59

Post by nathell »

sooheon wrote: post #56
nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

I wonder what disagreements or counterarguments downvoters might have, and if they might share them. For the parent, given what you've observed, do you still go to R for data crunching, or have you found anything in Clojure land that measures up?

R. Or a mixed approach, with Clojure for data preprocessing and R for the analysis proper. Case in point: I wrote the Clojure scraping library, Skyscraper [1], and made it output CSV by default so as to be able to easily drop the resulting files to R.

For statistics, Clojure has Incanter, but it's very basic in comparison. There are easily usable Java libraries for certain tasks (MALLET comes to mind), but these are few and far between.

[1]: https://github.com/nathell/skyscraper/

Re: R, the master troll of statistical languages (2012)

#60

Post by cvwright »

Fomite wrote: post #39
x0x0 wrote: post #12

Earlier quoted context omitted.

It's kind of amazing to see someone admit to spending hundreds or thousands of hours using R, yet refuse to spend a couple hours learning the language a little better. Whining that your tools are hard without investing any effort in them is just dumb. The R help even comes with code samples that you can run!

R is, I think, an interesting language because it's heavily used by people who would not otherwise learn a programming language. If you compare R not with other programming languages, but with other ways of working with statistical data, this makes far more sense. I don't actually "know" SAS in the way I know a programming language - I know the commands I invoke to do what I want it to do. Similarly, I encounter lots…

I see this as a failure of our educational system.

Speaking as an academic in CS, it's our job to teach people skills that they need for dealing with computers in the course of their career. The Math department does this for basic calculus and probability; the English department does this for literature and composition.

Why don't more CS departments offer the service courses that scientists and engineers need to really learn how to manipulate their data and make sense of it? At least part of the problem is probably that the other departments won't require their students to take such a course...

Post reply on HN