Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R: Introduction to Data Science (2019)

rafalab.dfci.harvard.edu

Post reply on HN
Print view View original

31–40 of 139 posts

Re: R: Introduction to Data Science (2019)

#31

Post by klmr »

acc_297 wrote: post #18
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

Pk/PD work for pharmaceutical data analysis I didn’t like using R at first but I’ve come to appreciate the speed that comes with months of experience. It’s a language which feels like it has a lot of magical incantations you need to remember - the default namespace is much more crowded. Functions like sapply vs mapply are tricky to reason about from the documentation alone. The values NA vs Null vs integer(0) are all…

> But in all honesty for the pharmaceutical industry it’s mostly momentum that keeps R on top

I can’t agree with this: especially in PK/PD, R is only just now taking over from the previous (closed-source) systems. Momentum would keep R out, not in.

Re: R: Introduction to Data Science (2019)

#32

Post by wodenokoto »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

Did you work with Rstudio Server and still found it not collaborative enough?

Re: R: Introduction to Data Science (2019)

#33

Post by wodenokoto »

hadley wrote: post #20
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

It’s been more than a few years since I worked in an R shop. While I loved wrangling and plotting data in the tidy verse I did find that the dependency management story in R to be even worse than Python.

Maybe that’s the problem?

Re: R: Introduction to Data Science (2019)

#34

Post by tropical333 »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

> What we found was that R had alot of packages but most haven't been touched in years and when you contact the owner you find they've often moved onto the python/pandas/scikit eco system

As a "bilingual" R & Python user, I've found this to be true for the latter language as well :)

I don't have much to add on top of what other useRs have mentioned, except another testimonial that our company has successfully used R in production for 6+ years, from data "pipeline" stuff you mentioned to dozens upon dozens of predictive models of varying complexities.

When faced with a new data analysis ask, 99%+ of the time I reach for R (although without the tidyverse, that number would be much lower). Like another commenter said, the ease by which you can plot in R blows Python away. Seaborn seems like a decent compromise in my limited experience, but plotting in "base" matplotlib makes me want to die.

Re: R: Introduction to Data Science (2019)

#35

Post by apwheele »

klmr wrote: post #30
zhdc1 wrote: post #16

Earlier quoted context omitted.

I’ve transitioned a lot of my work over to Julia, but R is still the most intuitive language I’ve used for scripting out data collection, cleaning, aggregation, and analysis cases. The ecosystem is simply better. The folks who maintain CRAN do a fantastic job. I can’t remember the last time a library incompatibility led to a show stopper. This is a weekly occurrence in Python.

> I can’t remember the last time a library incompatibility led to a show stopper. Oh, it’s very common unless you basically only use much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.

Agree with this, I am pretty agnostic to the pandas vs R whatever stuff (I prefer base R to tidyverse, and I like pandas, but realize I am old and probably not in majority based on comments online). But many teams who are "R adherent" folks I talk to are not deploying software in varying environments so much as reporting shops doing ad-hoc analytics.

For those whom want to use both R/python, I have notes on using conda for R environments, https://andrewpwheeler.com/2022/04/08/managing-r-environment....

Re: R: Introduction to Data Science (2019)

#36

Post by chollida1 »

hadley wrote: post #20
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

Wow, I really appreciate the reply. As I said in another comment here, I wish tidyverse was big when I was using R.

I was an R user from about 2003-2010.

We didn't have DPlyr at the moment though ggplot2 was coming around about that time I think. That helped alot for easy to develop visualizations.

But in our specific cases, the distributed libraries we used were written in python and integrated well with native python code. Pandas was just coming out around 2010, I think, and I think multi threading was also an issue then, but I can't really remember.

So our issues was partially our infrastructure tooling was going to python, but also we had a far easier time hiring people who were proficient in python and harder to find the same for R.

And once you start writing more code in python it starts to become harder to justify two separate code bases that can do the same thing so the R code got phased out and rewritten in python so we could have a single code base and not have to duplicate functionality in two languages.

Also a slight push for python came from the programmers who thought python represented a better language to know for their careers. Which looking back it does seem like python is used more often these days in general.

So I guess there isn't much you could have done in this case.

And as a side note, thanks for all the work you've done with R!!

Re: R: Introduction to Data Science (2019)

#37

Post by wodenokoto »

klmr wrote: post #30
zhdc1 wrote: post #16

Earlier quoted context omitted.

I’ve transitioned a lot of my work over to Julia, but R is still the most intuitive language I’ve used for scripting out data collection, cleaning, aggregation, and analysis cases. The ecosystem is simply better. The folks who maintain CRAN do a fantastic job. I can’t remember the last time a library incompatibility led to a show stopper. This is a weekly occurrence in Python.

> I can’t remember the last time a library incompatibility led to a show stopper. Oh, it’s very common unless you basically only use much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.

At my old job we snapshotted CRAN and pinned versions of package dependencies _against_ CRAN.

Re: R: Introduction to Data Science (2019)

#38

Post by nojito »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

>For those of you who us R still what is your use case?

Still the best replacement for EDA and reproducible analysis that used to be done in Excel.

Re: R: Introduction to Data Science (2019)

#39

Post by mushufasa »

hadley wrote: post #20
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

A few of the main issues I see, as a R user who built his company on python

- when we wanted to build a web app that processes data, it was a lot more straightforward to build both in python, so we can process data within the web servers instead of having to manage multiple stages of infrastructure and different languages. There's no Django for R.

- R will often do something instead of explicitly failing. This is the wrong tradeoff when running a production system, as if you're returning the wrong results to users you may not realize it unless there's an error

- R reproducible builds are worse than python. That's saying something because python is a pretty low bar. But running production systems you can't have builds suddenly fail week over week because one of a hundred packages was updated

Re: R: Introduction to Data Science (2019)

#40

Post by benreesman »

I'm looking at R seriously for the first time.

I've got a decade in with Python numeric computing, and I'm interested in Julia and all of the cutting-edge stuff.

I've only dabbled with R until now, and I haven't researched it enough to know if rumors of it's inevitable demise have any substance.

There are a lot of interesting math problems other than training gigantic neural networks on NVIDIA gear, and I've got some Computer Algebra System / ergonomic linear modeling needs on a current project:

I need the best tool for someone who is messing with Black-Scholes type stuff, who is still building the fidelity with tricky antiderivatives by hand, but I have enough fundamentals to check the computer's work.

What role should R play here?

Post reply on HN