R, the master troll of statistical languages (2012) • Hacker News Board

Re: R, the master troll of statistical languages (2012)

#101

Post by Gatsky » Wed, Feb 17, 2016, 3:26 AM UTC

Ok, but this seems pretty trivial compared to the many exclusive advantages R has. I've had minimal problems using and extending other people's software packages written in R (for bioinformatics). This has definitely not been the case with Java, R or perl, where just installing said software package is often unusually painful or impossible.

I think R is a prime example how useful a domain specific language can be. As such, I see Julia as the most viable replacement, although that will take a long, long time.

Re: R, the master troll of statistical languages (2012)

#102

Post by platz » Wed, Feb 17, 2016, 3:35 AM UTC

zhte415 wrote: post #92

cvwright wrote: post #60

Earlier quoted context omitted.

I see this as a failure of our educational system. Speaking as an academic in CS, it's our job to teach people skills that they need for dealing with computers in the course of their career. The Math department does this for basic calculus and probability; the English department does this for literature and composition. Why don't more CS departments offer the service courses that scientists and engineers need to real…

I'm sure there could be a very strong synergy with Economics and Finance. Especially the cross-over from CS to Finance.

Remember Javascript before ES5? 99% of "web programmers" didn't know the language either.

Re: R, the master troll of statistical languages (2012)

#103

Post by mojoe » Wed, Feb 17, 2016, 3:40 AM UTC

wanderfowl wrote: post #32

I'm a Post-Doc in a small social sciences department in a major university, and am probably the department's ranking R-geek. I did my dissertation, and much of my current work, doing modeling, analysis, and even machine learning in R. In many ways, I owe much of my success to the power that R has allowed me to wield. Multicore lapplys and ggplot2 are my life these days. But even with this, R drives me absolutely batt…

My data science group is currently transitioning to Python-based development from R. It's actually amazing how much faster Python development is, because of two things:

1. R libraries (and even rarely the R interpreter itself!) tend to have really weird corner case bugs that crop up every couple months, and

2. It's REALLY easy to write unmaintainable code in R, and so strange cruft creeps into the code over time.

The Python interpreter and Python statistical libs are rock solid in comparison, and with it we don't spend weeks debugging things caused by unnecessary idiosyncrasies. I just wish we'd started switching sooner and saving our time.

Re: R, the master troll of statistical languages (2012)

#104

Post by S4M » Wed, Feb 17, 2016, 3:46 AM UTC

avs733 wrote: post #86

wanderfowl wrote: post #32

I'm a Post-Doc in a small social sciences department in a major university, and am probably the department's ranking R-geek. I did my dissertation, and much of my current work, doing modeling, analysis, and even machine learning in R. In many ways, I owe much of my success to the power that R has allowed me to wield. Multicore lapplys and ggplot2 are my life these days. But even with this, R drives me absolutely batt…

I am in an almost identical situation, just still in the PhD stage. I moved from Matlab (originally a mechanical engineer) and the biggest shock was documentation and just the internal help stuff in general. The help files on a regular basis require you to understand how something works to understand the thing explaining how it works. I am often just shocked at little quirks I find trying to do things in R, not that…

> Oh and 'It's true that '<-' is a strange choice, but you can use '=' for variable assignments as well.

Re: R, the master troll of statistical languages (2012)

#105

Post by hadley » Wed, Feb 17, 2016, 3:51 AM UTC

hudibras wrote: post #98

tmalsburg2 wrote: post #62

Writing a variant of this article has become a rite of passage for all serious users of R. There are two issues that contribute to the difficulties people experience with R. First, yes, R can be confusing at times. Tal explains this really well, but only scratches the surface. There is so much more confusing and counter-intuitive stuff, for example with regards to factors that only very few people seem to understand…

I agree with everything here, including the praise for Matloff's book; it should be the very first book any serious R user picks up. But Matloff is no longer alone: Hadley Wickham's Advanced R is now also a must-read for R programmers. http://www.amazon.com/Advanced-Chapman-Hall-CRC-Series/dp/14...

And Advanced R is also available for free at http://adv-r.had.co.nz/

Re: R, the master troll of statistical languages (2012)

#106

Post by hadley » Wed, Feb 17, 2016, 3:52 AM UTC

noelwelsh wrote: post #30

I think the general consensus is that R is a terrible language with a lot of useful libraries. I especially like that R claims to be inspired by Scheme, but the memo seems to have been "Make sure we f *ck this all up" taped to the front of the "Lambda the Ultimate" papers[1]. In particular, lexical scoping was one of the key innovations in Scheme and R has pervasively buggered up their implementation, from not distin…

No, that is not the consensus. R is (at its heart) a beautiful language that is extremely well suited to its domain. Most people who use R are not professional programmers (or even identify as programmers) so it's not surprising that there's a lot of bad code written in R.

Re: R, the master troll of statistical languages (2012)

#107

Post by hadley » Wed, Feb 17, 2016, 4:01 AM UTC

x0x0 wrote: post #44

chrisseaton wrote: post #41

Earlier quoted context omitted.

Why can't a JIT solve this? It shouldn't need to do the bookkeeping for every iteration if it has JIT compiled it. A JIT should be able to take advantage of processor vector instructions etc.

There's some movement in that direction. However, the R core committers are essentially not only volunteers, but they're all (afaik) academic statisticians. One of the people who made strides in this direction is primarily an computational statistician at Iowa (Luke Tierney / compiler package). Building a high performance runtime/jit is wildly out of their scope of expertise. In retrospect, and I think many of them w…

I think, and I'm pretty sure most of R core would agree, that building and maintaining their own runtime _was_ the right thing to do. Otherwise R would have been at mercy of maintainers who were interested in problems other than creating an expressive language for data analysis.

Re: R, the master troll of statistical languages (2012)

#108

Post by hadley » Wed, Feb 17, 2016, 4:02 AM UTC

nathell wrote: post #66

jonchang wrote: post #57

Earlier quoted context omitted.

See, this is another interesting example of the kind of behavior described here: https://news.ycombinator.com/item?id=11113042 People who don't take the time to learn the language are having to go through these contortions to make R work the way their favorite language works, rather than just taking the time to learn how R works! R has a useful function, paste, that concatenates strings together. Only it takes vararg…

Thanks for pointing this out. I overlooked it, presumably because it's in the last paragraph of "Details" and not illustrated in any example. I still maintain there's a wart in what I'd described, which is `do.call` not accepting vectors as the second argument. Also, `collapse` is idiosyncratic: I have to remember a special knob for every function that has a vararg and non-vararg flavour. You raise the point of takin…

You might enjoy purrr, https://github.com/hadley/purrr, which is my attempt to make FP tools in R more consistent.

Re: R, the master troll of statistical languages (2012)

#109

Post by tanvach » Wed, Feb 17, 2016, 5:15 AM UTC

nathell wrote: post #48

I have a love-hate relationship with R, being a predominantly Clojure (and Ruby these days) programmer who only occassionally dabbles in data crunching. The apply/sapply dichotomy that the article mentions (actually a hexachotomy, there are also mapply, sapply, tapply and vapply) is one example of a gazillion warts that the language has. Another random one: R has a useful function, paste, that concatenates strings to…

You can also use paste( v, collapse =",")

Re: R, the master troll of statistical languages (2012)

#110

Post by z92 » Wed, Feb 17, 2016, 5:48 AM UTC

klmr wrote: post #74

Cogito wrote: post #71

Earlier quoted context omitted.

Any idea where the speedups came from? Is it that the problems weren't algorithmically limited in the first place (lots of io for example), reduction of overhead etc (what kind of python was the code running on before?), or just that the speedup on low level operations added up cumulatively and cam e to dominate the other timing factors? Also, did you change the data structures or use the same ones as in python? Was…

Python and similar dynamic languages suffer from the fact that every name access (variable, function, etc) incurs a dynamic lookup of that name in a (nested) dictionary. Statically compiled languages don’t have this. There are fairly recent, clever optimisations that can avoid many of these lookups but (a) they are not implemented in any of the common implementations of Python, R, etc (JavaScript has them though). Bu…

The lookup thing only happens during compilation to byte code or intermediate code, I believe. Once in byte code, there are no variable names, only addresses.