Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R Passes SAS in Scholarly Use

r4stats.com

Post reply on HN
Print view View original

21–30 of 126 posts

Re: R Passes SAS in Scholarly Use

#21

Post by jacobolus »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

The even bigger advantage of python is everything else: web scraping, interfacing with weird APIs, hitting the database, consuming and emitting obscure formats, parsing text, calling operating system services, and a million other things. That and much much better abstractions for building larger systems out of reusable components and much better tooling for serious software engineering.

R or Matlab can be fine at exploratory data analysis, making charts, running miscellaneous bits of non-programmer-grad-student code found on the internet, etc., but as soon as you want to do anything other than data analysis, they quickly become annoying.

Re: R Passes SAS in Scholarly Use

#22

Post by nol13 »

uptownfunk wrote: post #13

Some reasons I love / use R: Plenty of free high quality documentation and learning materials around R (just read anything by Hadley) Package manager. Super easy to find, install, and start using packages. Open source / Free Large community of users Extensive usage by the stats community. (If a new algorithm comes out, chances R there will be an R implementation) Easy to build and share your own packages via Github.…

Basically this. Certainly not the worst language (SAS), but intangibly less pleasurable than python or most other common languages I've used. Maybe that's because I'm not a 'real' stats person though and ~ notation still takes me a minute to grok. And there is no challenger on he horizon for it's dominance in available packages. And at least it's not SAS.

My theories:

A) Stat people use R because because they don't know any better B) Stat people use R because code monkeys use python C) Stat people use R because stat people use R (probably this)

Re: R Passes SAS in Scholarly Use

#23

Post by minimaxir »

nol13 wrote: post #22
uptownfunk wrote: post #13

Some reasons I love / use R: Plenty of free high quality documentation and learning materials around R (just read anything by Hadley) Package manager. Super easy to find, install, and start using packages. Open source / Free Large community of users Extensive usage by the stats community. (If a new algorithm comes out, chances R there will be an R implementation) Easy to build and share your own packages via Github.…

Basically this. Certainly not the worst language (SAS), but intangibly less pleasurable than python or most other common languages I've used. Maybe that's because I'm not a 'real' stats person though and ~ notation still takes me a minute to grok. And there is no challenger on he horizon for it's dominance in available packages. And at least it's not SAS. My theories: A) Stat people use R because because they don't k…

D) Stat people use R and Python, depending on what best fits the use case.

Re: R Passes SAS in Scholarly Use

#24

Post by minimaxir »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

> All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data frames, Java's ArrayLists, etc.

At the end of the day, for machine learning applications, your data is in a tabular format. (in Python, a pandas data frame) Yes, Python has a few tricks like list comprehensions for speeding up data processing into that analyzable form. R has a few tricks for processing tabular data as well. (e.g. dplyr).

There are tradeoffs and the skill is finding which works best. Using a single programming language is a bad philosophy even for non-statistical developers.

Re: R Passes SAS in Scholarly Use

#25

Post by minimaxir »

jacobolus wrote: post #21
aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

The even bigger advantage of python is everything else : web scraping, interfacing with weird APIs, hitting the database, consuming and emitting obscure formats, parsing text, calling operating system services, and a million other things. That and much much better abstractions for building larger systems out of reusable components and much better tooling for serious software engineering. R or Matlab can be fine at ex…

If you haven't looked at the R ecosystem in awhile, there is a package for each of those use cases. (scraping/API: rvest; database: dplyr; parsing text: stringr, etc).

Yes, Hadley Wickham is primarily responsible for the popularity of R.

Re: R Passes SAS in Scholarly Use

#26

Post by nol13 »

minimaxir wrote: post #23
nol13 wrote: post #22

Earlier quoted context omitted.

Basically this. Certainly not the worst language (SAS), but intangibly less pleasurable than python or most other common languages I've used. Maybe that's because I'm not a 'real' stats person though and ~ notation still takes me a minute to grok. And there is no challenger on he horizon for it's dominance in available packages. And at least it's not SAS. My theories: A) Stat people use R because because they don't k…

D) Stat people use R and Python, depending on what best fits the use case.

Ya but the only use case for R over python is because some package is written in it.

Re: R Passes SAS in Scholarly Use

#27

Post by mziel »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

I think R's data frames is by far the most intuitive abstraction. There's a reason why it was duplicated in Python, Spark or Julia. And R has a lot of utilities for converting input strings, dates, JSON etc.

Re: R Passes SAS in Scholarly Use

#28

Post by gaius »

One thing that interests me is language power, vs experience. Let's say you had 1 year experience in language X. Language Y comes along that is better in some way. In another year, would you be happier and more productive with 2 years experience of X, or one year of Y?

I sometimes think with the churn of languages, no-one really gets deeply enough into one to really leverage it.

Re: R Passes SAS in Scholarly Use

#29

Post by kpil »

LeifCarrotson wrote: post #19
rsrsrs86 wrote: post #16

Am I right to believe that there is no way that proprietary scientific software can keep up with open source?

No. Some have the benefit of proprietary modules (FPGA toolchains), some have large libraries of pre-entered and organized data (Mathematica), and some have early access to hardware (LabView, CUDA). Programming languages, perhaps, are less vulnerable to these issues. And perhaps open source could beat these applications eventually, given perfect competition. But we're not in that world, unfortunately.

I think that open source programming languages will always win in the long run, since the target customer base knows how to program and extend the tools.

Re: R Passes SAS in Scholarly Use

#30

Post by kensai »

The success of R in Statistics (in respect to Python, etc) was that it was thought from the beginning with Statisticians and their specific needs and approaches in mind. As much as I appreciate Python, it is a general purpose programming language adapted to Statisticians needs, not the other way around.

R has many issues, but if you speak to Statisticians you will hear that its the closest thing they have to their own way of doing things.

Post reply on HN