Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R, the master troll of statistical languages (2012)

talyarkoni.org

Post reply on HN
Print view View original

61–70 of 154 posts

Re: R, the master troll of statistical languages (2012)

#61

Post by 50CNT »

TheLogothete wrote: post #16
mziel wrote: post #4

The problem is people using R without trying to learn about the language itself, just assuming it works like their favourite language. For example complaining that R is slow and then writing iterative solution instead of using vectorization. When I saw the example the author gave my first thought was "sapply/lapply". Lapply is essential to the R use, and is being taught early on in every book/course on R I've ever sa…

In my experience a lot of the claims that R is slow are greatly exaggerated and made by people who don't actually use it. Kind of an echo chamber. Every time I see someone say chose Python instead because of speed, I roll my eyes.

I for one write all my statistical code in baremetal assembly. I manage about 5 a year, but they all run very quickly. There is no such thing as premature optimization.

Re: R, the master troll of statistical languages (2012)

#62

Post by tmalsburg2 »

Writing a variant of this article has become a rite of passage for all serious users of R. There are two issues that contribute to the difficulties people experience with R. First, yes, R can be confusing at times. Tal explains this really well, but only scratches the surface. There is so much more confusing and counter-intuitive stuff, for example with regards to factors that only very few people seem to understand fully. However, there is a second issue, and this is less often acknowledged: People expect R to be immensely powerful and at the same time easy to use, which is really not a very reasonable thing to expect. This attitude is fairly specific to the R community. No C++ developer would dare to write a long rant about the shortcomings of C++ while at the same time nonchalantly admitting that they never made a serious attempt to learn it. One symptom of this problem is that hardly any self-proclaimed R hacker has read Matloff's book "The Art of R Programming" which was for a long time, and perhaps still is, the only book on R programming. The mere fact that there is (or was) only one such book speaks volumes.

Re: R, the master troll of statistical languages (2012)

#63

Post by DangerousPie »

I have my fair share of problems with R, but that first example (4 ways to select a column) seems a bit silly. Just off the top of my head, I could think of plenty of ways to do the same thing in Python/pandas:

    ice_cream.icol[0]
    ice_cream['col']
    ice_cream.iloc[:, 0]
    ice_cream.loc[:, 'col']
    ice_cream.ix[:, 'col']
And if you wanted to make things more convoluted, you could also wrap things into lists like the author did in the R example. So this is definitely not a problem that is unique to R or any reasonably flexible language.

Re: R, the master troll of statistical languages (2012)

#64

Post by [deleted] »

sooheon wrote: post #56
nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

I wonder what disagreements or counterarguments downvoters might have, and if they might share them. For the parent, given what you've observed, do you still go to R for data crunching, or have you found anything in Clojure land that measures up?

[deleted]

Re: R, the master troll of statistical languages (2012)

#65

Post by nothis »

wanderfowl wrote: post #32

I'm a Post-Doc in a small social sciences department in a major university, and am probably the department's ranking R-geek. I did my dissertation, and much of my current work, doing modeling, analysis, and even machine learning in R. In many ways, I owe much of my success to the power that R has allowed me to wield. Multicore lapplys and ggplot2 are my life these days. But even with this, R drives me absolutely batt…

I just wonder if there's rationality behind not changing this (by either switching to a different language, fixing it or, hardcore mode, create a new one).

I know from a very, very different field that you often have to deal with decade(s) old technology because your employee/professor/etc is just used to it and, 15 years ago, it simply was the best option. I guess it's an equation that puts time spent learning its quirks vs time saved using a more sensible tool. While tiresome, "learning" might actually be the faster way to get things done. But it also carries so much ballast that, if there's a better alternative, must waste millions of frust-hours (and actual errors), especially for newcomers who could just as well learn a new tool - faster.

Re: R, the master troll of statistical languages (2012)

#66

Post by nathell »

jonchang wrote: post #57
nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

See, this is another interesting example of the kind of behavior described here: https://news.ycombinator.com/item?id=11113042 People who don't take the time to learn the language are having to go through these contortions to make R work the way their favorite language works, rather than just taking the time to learn how R works! R has a useful function, paste, that concatenates strings together. Only it takes vararg…

Thanks for pointing this out. I overlooked it, presumably because it's in the last paragraph of "Details" and not illustrated in any example.

I still maintain there's a wart in what I'd described, which is `do.call` not accepting vectors as the second argument. Also, `collapse` is idiosyncratic: I have to remember a special knob for every function that has a vararg and non-vararg flavour.

You raise the point of taking the time to learn the language, and I acknowledge this. Yet, as an occassional user, this is precisely what I'd like to avoid. When working with R, I'm pragmatic: what I'm after is a working solution to the problem at hand, rather than its most succinct or elegant formulation. When I find one, I move on. In production code this would incur a technical debt, but due to R's exploratory nature, this is typically not much of a problem. Had the language been more consistent, it would take less time to learn it thoroughly.

Re: R, the master troll of statistical languages (2012)

#67

Post by mackwic »

justin_oaks wrote: post #9

Way too many languages are trolls, or have troll features. Or in other words, too many languages have features that don't do what a reasonable person would expect them to do. I've long considered implicit type conversion to be a troll feature, especially how Javascript does it. Another one is how differently Java treats primitives and Object types. Oracle databases treat nulls and empty strings the same. At times lik…

You can take a look at Rust. The authors are really careful to design an elegant C++ replacement, with no troll features and zero-cost abstractions.

Re: R, the master troll of statistical languages (2012)

#68

Post by capnrefsmmat »

My biggest complaint about R isn't the inconsistency and obtuseness -- I've been using it long enough to get familiar with the documentation and the zillions of varieties of apply. My problem is the data structures.

R has only a few core data structures: vectors, lists, arrays, and matrices. Data frames are built on top of lists, and admittedly data frames are incredibly useful for statistics -- there's a reason pandas exists, and a reason data analysis is much more tedious in other languages.

But there are no hash maps or sets (lists have named elements, but with O(n) indexing; the only hash tables available use environments and accept limited types of keys), no tuples, no structural or record types, stacks and queues only recently became available on CRAN (through C), and so on.

This leads to the folk belief that the only way to optimize R is to vectorize code or to write in in C or C++ (with Rcpp, for instance). No statistical programmer ever thinks about choosing the right data structure for the job, since you basically only ever use lists and data frames. Fast operations on data structures (like graph algorithms) have to be written in C. There's just no way to do it in R.

When I co-taught a statistical computing course, covering the basics of data structures and algorithms, I included some homework assignments where the difference between a fast and a slow algorithm was the choice of data structure. R users struggled because they had very little available to them. If their code wasn't fast because they were doing O(n) list lookups in a loop, there wasn't anything they could do to fix it.

I hope Python and Julia can eat R's lunch. Some day I'll have to get around to trying Julia for a serious project...

Re: R, the master troll of statistical languages (2012)

#69

Post by GFK_of_xmaspast »

existencebox wrote: post #33
TheLogothete wrote: post #16

Earlier quoted context omitted.

In my experience a lot of the claims that R is slow are greatly exaggerated and made by people who don't actually use it. Kind of an echo chamber. Every time I see someone say chose Python instead because of speed, I roll my eyes.

It's funny you bring up python; I say this not as a comment on your thesis, but related, since I often hear the "python is slow" trope but that's only half true, you can typically write python that is plenty "fast enough" (As a day-jobbing data pipeline engineer) if you're implementing with an understanding of what things will drive you into the mud. This goes beyond just understanding the tool you're using, fundamen…

I've translated plenty of numerical code from (pure-ish) python to c and c++, and usually get about a 100x speedup, sometimes as high as 800x, implementing the same algorithms.

Re: R, the master troll of statistical languages (2012)

#70

Post by GFK_of_xmaspast »

bachmeier wrote: post #55
superuser2 wrote: post #5

Most of my university classmates' first exposure to programming is using R in a statistics class. It's awful. I wish they'd make Python or something a prerequisite, so that giant swaths of people don't get turned off of computing or start with the strange ideas it teaches.

It sounds like you are arguing for imperative programming over functional programming.

What does that have to do with r vs python?
Post reply on HN