Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R Passes SAS in Scholarly Use

r4stats.com

Post reply on HN
Print view View original

31–40 of 126 posts

Re: R Passes SAS in Scholarly Use

#31

Post by blahi »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

If I use python I have to write half of the algorithms I use myself. Worse, I have to write a gazillion helper functions nyself. It just won't do

Re: R Passes SAS in Scholarly Use

#33

Post by michaelsbradley »

mziel wrote: post #27
aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

I think R's data frames is by far the most intuitive abstraction. There's a reason why it was duplicated in Python, Spark or Julia. And R has a lot of utilities for converting input strings, dates, JSON etc.

data.table and dplyr take data frames up a notch!

https://github.com/Rdatatable/data.table/wiki/Getting-starte...

https://cran.rstudio.com/web/packages/dplyr/vignettes/introd...

xts is quite useful as well, when you need to do modeling/analysis in an especially time-oriented manner:

http://joshuaulrich.github.io/xts/

Re: R Passes SAS in Scholarly Use

#35

Post by mikeskim »

The way I use Python in machine learning is quite different from how many others in competitive ML use Python. I use Python purely for Python 2.7 with Pypy and try not to touch or use numpy,scipy,pandas,etc. R's data.table is possibly faster than Python's numpy/scipy/pandas. I think anyone claiming Python because of numpy/scipy/pandas is really being mislead. You should be using Python in spite of the need to rely upon numpy/scipy/pandas. If you really need numpy/scipy/pandas just use R and data.table which is amazingly fast. I think Python is really great because of Pypy and the strength of the standard Python library.

Re: R Passes SAS in Scholarly Use

#36

Post by jacobolus »

minimaxir wrote: post #25
jacobolus wrote: post #21

Earlier quoted context omitted.

The even bigger advantage of python is everything else : web scraping, interfacing with weird APIs, hitting the database, consuming and emitting obscure formats, parsing text, calling operating system services, and a million other things. That and much much better abstractions for building larger systems out of reusable components and much better tooling for serious software engineering. R or Matlab can be fine at ex…

If you haven't looked at the R ecosystem in awhile, there is a package for each of those use cases. (scraping/API: rvest; database: dplyr; parsing text: stringr, etc). Yes, Hadley Wickham is primarily responsible for the popularity of R.

We’re talking about an order of magnitude difference in number of packages (82096 on PyPI vs. 8551 on CRAN) and their maturity, and such a naïve metric probably undersells the difference in variety of use cases.

If you picked 100 random production python projects out of a hat, no more than a small handful of them would be remotely appropriate to build using R.

And that's entirely fine. R is great at being a quick and dirty statistical analysis language for people doing data exploration.

Re: R Passes SAS in Scholarly Use

#37

Post by Hondor »

rsrsrs86 wrote: post #16

Am I right to believe that there is no way that proprietary scientific software can keep up with open source?

For simulation it's not even close. Open source can't do user interfaces. Not a problem for programming languages, but for simulation at least, there's a slew of powerful but unusable open source software made by professors and then there's expensive proprietary ones with nice front ends that save enormous amount of time for the users.

Re: R Passes SAS in Scholarly Use

#38

Post by whyever »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

R has the best plotting library (ggplot) by far.

Re: R Passes SAS in Scholarly Use

#39

Post by Myrmornis »

blahi wrote: post #31
aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

If I use python I have to write half of the algorithms I use myself. Worse, I have to write a gazillion helper functions nyself. It just won't do

I think you got downvoted because people didn't know what you meant. You're right though. Although R is laughably inferior to python as a programming language, it is vastly more work to try to do statistical data analysis in python than in R. I recommend using both languages and using csv or whatever format to exchange data sets.

Re: R Passes SAS in Scholarly Use

#40

Post by compactmani »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

Can you give some concrete examples of the benefits of wrangling in python over R and a general sense of how much time this would save a user?
Post reply on HN