Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R Passes SAS in Scholarly Use

r4stats.com

Post reply on HN
Print view View original

71–80 of 126 posts

Re: R Passes SAS in Scholarly Use

#71

Post by carlmcqueen »

Myrmornis wrote: post #39
blahi wrote: post #31

Earlier quoted context omitted.

If I use python I have to write half of the algorithms I use myself. Worse, I have to write a gazillion helper functions nyself. It just won't do

I think you got downvoted because people didn't know what you meant. You're right though. Although R is laughably inferior to python as a programming language, it is vastly more work to try to do statistical data analysis in python than in R. I recommend using both languages and using csv or whatever format to exchange data sets.

feather is a great way to move data between the two as it is blazingly fast and native in python and R.

Re: R Passes SAS in Scholarly Use

#72

Post by dekhn »

EDIT: I am corrected in regards to the SAS routines statement; see the reply.

A few comments. I worked in pharma and the FDA specifically requires a number of SAS routines- specific function calls- to be used when doing drug studies/clinical trials. R can't replace SAS in those cases without massive effort because the FDA is slow and conservative and people like to have validated results.

I think the writing was on the wall for SAS when this article came out: http://www.nytimes.com/2009/01/07/technology/business-comput...

The SAS spokesperson said: """Anne H. Milley, director of technology product marketing at SAS. She adds, “We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”"""

to which a senior employee of Boeing pointed out that every jet they build uses R as an integral part of the design process. I think that had to be an "oh shit" moment for SAS, where they realized their strong position in stats was going to start to erode.

Re: R Passes SAS in Scholarly Use

#73

Post by jiiam »

Good.

* Rant mode: On

Maybe in 30 years they will also learn a true programming language and stop producing undocumented, unusable, unportable, underdeveloped libraries for research level tools and technologies.

Outside the world of Neural Network it is a complete disaster, and the NN landscape is at an acceptable level only because of big companies, surely not thanks to the researchers. And the reason, of course, is that most researchers refuse to think of themselves as "software developer" and use these arcane languages which might be good for prototyping but lack power when it comes to shipping a real product (which might also be a tool for other researchers to use).

At least they're not using Matlab where everything breaks as soon as you change machine.

* Rant mode: Off

Re: R Passes SAS in Scholarly Use

#74

Post by minimaxir »

ekianjo wrote: post #50
minimaxir wrote: post #25

Earlier quoted context omitted.

If you haven't looked at the R ecosystem in awhile, there is a package for each of those use cases. (scraping/API: rvest; database: dplyr; parsing text: stringr, etc). Yes, Hadley Wickham is primarily responsible for the popularity of R.

Dplyr is not for databases at all. Its a piping operator that simplifies complex syntax a lot.

Yes, you can use dplyr to connect and query data from a database. https://cran.r-project.org/web/packages/dplyr/vignettes/data...

Re: R Passes SAS in Scholarly Use

#75

Post by Houshalter »

jacobolus wrote: post #36
minimaxir wrote: post #25

Earlier quoted context omitted.

If you haven't looked at the R ecosystem in awhile, there is a package for each of those use cases. (scraping/API: rvest; database: dplyr; parsing text: stringr, etc). Yes, Hadley Wickham is primarily responsible for the popularity of R.

We’re talking about an order of magnitude difference in number of packages (82096 on PyPI vs. 8551 on CRAN) and their maturity, and such a naïve metric probably undersells the difference in variety of use cases. If you picked 100 random production python projects out of a hat, no more than a small handful of them would be remotely appropriate to build using R. And that's entirely fine. R is great at being a quick and…

I suspect that the utility of more packages increases only logarithmically. Having 10x more packages doesn't mean it's 10 times more useful. If your obscure need isn't in the first 8,000 packages, it probably won't be in the next 80,000. That's just how power laws work. And any common task you can think of will probably be in the top 8,000.

Re: R Passes SAS in Scholarly Use

#76

Post by hadley »

dekhn wrote: post #72

EDIT: I am corrected in regards to the SAS routines statement; see the reply. A few comments. I worked in pharma and the FDA specifically requires a number of SAS routines- specific function calls- to be used when doing drug studies/clinical trials. R can't replace SAS in those cases without massive effort because the FDA is slow and conservative and people like to have validated results. I think the writing was on t…

That is not true. The FDA uses R internally, and there is no requirement that you must use any specific software tool. See https://www.r-project.org/doc/R-FDA.pdf for more details

Re: R Passes SAS in Scholarly Use

#77

Post by hadley »

Houshalter wrote: post #75
jacobolus wrote: post #36

Earlier quoted context omitted.

We’re talking about an order of magnitude difference in number of packages (82096 on PyPI vs. 8551 on CRAN) and their maturity, and such a naïve metric probably undersells the difference in variety of use cases. If you picked 100 random production python projects out of a hat, no more than a small handful of them would be remotely appropriate to build using R. And that's entirely fine. R is great at being a quick and…

I suspect that the utility of more packages increases only logarithmically. Having 10x more packages doesn't mean it's 10 times more useful. If your obscure need isn't in the first 8,000 packages, it probably won't be in the next 80,000. That's just how power laws work. And any common task you can think of will probably be in the top 8,000.

I totally buy that argument, but there's probably not a huge overlap in the set of "common tasks" for R and python.

Re: R Passes SAS in Scholarly Use

#78

Post by hadley »

roel_v wrote: post #62
ldp01 wrote: post #51

Earlier quoted context omitted.

There are many things wrong with R but basic plotting functions are one of its strengths. Is this the way you did it? It seems pretty intuitive... a=pi/180 x=1:360 plot(x,sin(a * x)) plot(x,a * cos(a * x)) plot(x,-a^2 * sin(a * x))

But that just draws 3 separate plots. My main problem was the derivative, not so much the plotting (or maybe it was 'plotting an arbitrary function'); but I looked it up and it seems I slightly misremembered what it was I wanted to do. I wanted to draw a cubic spl ine, not a s ine. What I ended up doing was spline_x I still don't quite understand how that derivative works - ?list doesn't mention anything about 'deriv…

It took me 5 minutes to figure out how that actually did work! That is rather esoteric code!

(FWIW the reason that there's no native support in ggplot2 for this sort of smoothing is that I think it's a really bad idea as it tends to distort the underlying data)

Re: R Passes SAS in Scholarly Use

#79

Post by hadley »

jhbadger wrote: post #55
pollitos wrote: post #41

R is really LISP with syntactic sugar and bindings to well respected high-performance FORTRAN matrix and math optimization codes. http://librestats.com/2011/08/27/how-much-of-r-is-written-in... It's great for bleeding edge scientific research. The results of many languages don't always match for advanced algorithms, but the open source nature of R, makes it easier to identify the problem areas. The R-core interpreter…

People say that, but I'd prefer the actual LISP syntax then (being a fan of xlispstat back in the day). I'm surprised nobody has created a "Lisp-flavored R" analogous to Erlang's LFE or Python's Hy.

Most R users aren't programmers, and a LISP-y interface, while powerful, tends to be intimidating.

Re: R Passes SAS in Scholarly Use

#80

Post by micro_cam »

sillysaurus3 wrote: post #5

Mathematica isn't being used at all? That's surprising. Mathematica is wonderful. I wonder what's holding it back? It doesn't seem to have a package manager. Could it be that simple?

It's been about 10 years since I looked at mathematica but at that time it put the emphasis on symbolic manipulation of equations using its own internal magic while R (and matlab and numpy) focus on more traditional numerical computation, eg array operations via BLAS/LAPACK which is much more practical for statistics.
Post reply on HN