Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R Passes SAS in Scholarly Use

r4stats.com

Post reply on HN
Print view View original

101–110 of 126 posts

Re: R Passes SAS in Scholarly Use

#101

Post by _of »

Malarkey73 wrote: post #68

I'm not totally sure whether this analysis captures the true extent which R vs SAS vs SPSS is used. If I use R for a plot, or a simple bit of regression, or anova, or even cross-validation. I don't reference it in a paper. I only cite it if there is a package designed for a particular type of data (e.g. a Bioconductor package) or something a bit more esoteric (e.g. apcluster). About 95% of the work is data munging an…

I don't cite packages like plyr and ggplot, but I do cite what program I use for statistics (R, etc) and its version.

Re: R Passes SAS in Scholarly Use

#102

Post by noelsusman »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

List comprehension is nice, but Hadley's packages for R blow Python out of the water when it comes to data wrangling.

Personally, I think anyone who works with data extensively should be familiar with both Python and R.

Re: R Passes SAS in Scholarly Use

#103

Post by noelsusman »

jiiam wrote: post #73

Good. * Rant mode: On Maybe in 30 years they will also learn a true programming language and stop producing undocumented, unusable, unportable, underdeveloped libraries for research level tools and technologies. Outside the world of Neural Network it is a complete disaster, and the NN landscape is at an acceptable level only because of big companies, surely not thanks to the researchers. And the reason, of course, is…

I mean, I won't argue against having better code and documentation, but it's not really our job to ship a real product. Shipping well documented, easily usable, ultra portable, well developed libraries takes a fuckload of time, resources, and expertise that we don't have. Our primary job is to ship ideas.

It would be awesome if every project I did ended up with a nice, polished piece of software, but that's not what I get paid to do. I would be fired if I tried to do that.

Re: R Passes SAS in Scholarly Use

#104

Post by chrisamiller »

hooloovoo_zoo wrote: post #92
Malarkey73 wrote: post #68

I'm not totally sure whether this analysis captures the true extent which R vs SAS vs SPSS is used. If I use R for a plot, or a simple bit of regression, or anova, or even cross-validation. I don't reference it in a paper. I only cite it if there is a package designed for a particular type of data (e.g. a Bioconductor package) or something a bit more esoteric (e.g. apcluster). About 95% of the work is data munging an…

Why don't you cite the packages you use?

Our genomics workflows use dozens of packages even before I get the data and start really doing analysis, statistics, and plots. It's just not feasible to cite every bit of code we use (though we certainly point people to the higher level routines, which they can use to see what was run/how to reproduce the results).

Re: R Passes SAS in Scholarly Use

#105

Post by blahi »

jnbiche wrote: post #67
blahi wrote: post #52

Earlier quoted context omitted.

How pandas solves statistical problems or the thousand utility functions that are missing? Please, explain.

Have you even looked at pandas? I get the impression you haven't. Pandas has most of the statistical utility functions that R does, and for those few that it lacks, Python/Pandas also has available an easy-to-use FFI interface to R via rpy2. Take a look: http://pandas.pydata.org/pandas-docs/stable/ EDIT: Specified name of interface (rpy2).

Have you even looked at statistics? I get the impression you haven't. Pandas has aggregation and that's it. I don't really like to start those language wars, but what the heck. Your precious pandas is actually worse than the R alternatives in every way. It is slower than data.table while having comparable syntax. Dplyr is comparable in speed while having WAY better syntax.

Re: R Passes SAS in Scholarly Use

#106

Post by chrisamiller »

griffinmichl wrote: post #93
jsprogrammer wrote: post #89

Earlier quoted context omitted.

Shouldn't your full source be available; which would explicitly record your dependencies?

Source code really should be available, but it almost never is. The peer review process is, in my opinion, quite flawed. While your paper's high level content gets reviewed, no one actually looks at your code and data to ensure that you didn't forget to carry the one. Your analysis could be totally wrong, but reviewers only review what you say you did, not what you actually did.

Any paper I peer review better have source code available or they will hear about it. That said, yes, there isn't time (or funding) to actually re-run the entire analysis.

Re: R Passes SAS in Scholarly Use

#107

Post by dbs »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

I use both Python and R for analytics projects. Python falls quite short on statistics and time series, especially if one is doing exotic/advanced stuff.

By the end of the day I just want to get my job done, and I select the best tool for the job. I have gained a lot by working with both languages.

On a sidenote, one cannot talk about the success of R without mentioning RStudio (amazing IDE for working with data)

Re: R Passes SAS in Scholarly Use

#108

Post by baldfat »

aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

Python the second best option for just about everything.

R has a wonderful ecosystem that does amazing data manipulation (dplyr for example)

Python is a good choice but it isn't "clearly better."

Re: R Passes SAS in Scholarly Use

#109

Post by baldfat »

hadley wrote: post #77
Houshalter wrote: post #75

Earlier quoted context omitted.

I suspect that the utility of more packages increases only logarithmically. Having 10x more packages doesn't mean it's 10 times more useful. If your obscure need isn't in the first 8,000 packages, it probably won't be in the next 80,000. That's just how power laws work. And any common task you can think of will probably be in the top 8,000.

I totally buy that argument, but there's probably not a huge overlap in the set of "common tasks" for R and python.

Pandas for Python is almost re-imaging of R (Base 0 instead of base 1 UGH) very closely and the "common task" match closely, but I still use R over Python.

Re: R Passes SAS in Scholarly Use

#110

Post by baldfat »

ldp01 wrote: post #47
aabajian wrote: post #15

I think Python is the biggest hidden gem in statistics. It's had a tremendous impact on machine learning and algorithm development, yet traditional statisticians still rely on SAS/R/Stata/MATLAB. All of these languages have libraries that produce the same results, the difficulty is mangling the data into the correct input format. Python's list comprehensions are much, much easier to use than MATLAB matrices, R's data…

I'm glad I read this comment. After checking some of the docs I think I will have a go at Python for data wrangling. List comprehensions look... friendly. R still rules for plotting and running canned statistical procedures but sometimes I feel like if I stop programming R for a week I forget how to use it effectively... E.g. Forgetting to add stringsAsFactor=FALSE to everything, forgetting rbind() can overwrite colu…

dplyr or data.frame is THE DATA MANIPULATION library just like ggplot2 is THE PLOTTING program for all other languages.
Post reply on HN