Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R: Introduction to Data Science (2019)

rafalab.dfci.harvard.edu

Post reply on HN
Print view View original

41–50 of 139 posts

Re: R: Introduction to Data Science (2019)

#41

Post by vharuck »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

Why I still use R for analysis at work:

- R Markdown is just great for static reports. We use PowerBI or ArcGIS for interactive stuff.

- GIS is a breeze. My work provides licenses for ArcGIS, which has a Python library for scripting. Despite that, it is so much easier to do stuff in R, which can read and create ArcGIS shapefiles.

- Exploratory data analysis is easy. Often, before meetings, I'll connect to the database in R and make a few basic tables. Then I can query, aggregate, or plot data sitting the meeting. I have custom ggplot themes in a package, so even my happy hastily created plots look nice.

- RStudio is amazing. What it lacks in editing tricks, it more than makes up for in simplifying R-specific tasks. Showing plots is automatic, rendering and viewing markdown reports (of any type) is two buttons, testing and building a package are each two buttons.

- I spent a lot of time evangelizing R (team-wide presentations, being the "R guy" for troubleshooting, organizing an R User Group with members from different teams, creating an internal package repository). Some became happy converts, the rest begrudgingly accepted it as a tool we would use. I don't know if I could do it again with another language.

I'll admit my work doesn't get incorporated into pipelines. We get the data, analyze it, create reports, and share the reports by email or on our public website. The statisticians are segregated from the developers here. State government resists change, especially role changes that don't match grants' or laws' wording.

Re: R: Introduction to Data Science (2019)

#42

Post by doodledoodahs »

hadley wrote: post #20
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

OK, since you're here!

(this all prefaced with a massive thank you for tidyverse, without which R is very crusty).

I love R for interactive work and quick analyses, but I'm currently trying to integrate various bits of R code into a large document-building pipeline and wishing I could use Python for it:

- Exception handling and error processing seem a pain in R. Maybe I'm doing it wrong, but if feels like a mess and not nearly as ergonomic as python. Trycatch seems to have gotchas related to scope because the error handling is in a function. The distinction between warning, stop etc seems odd. The option to stop on warnings isn't useful because older packages seem to abuse warnings as messages. I have just discovered `safely` which is helpful, but then you have to unwrap lists in pipelines which feels clunky.

- Related, I _really_ wish we could just drop model objects or other tibbles as single objects directly into a tibble cell rather than as list(df). Unpacking lists and checking objects inside them exist is much more of a pain (e.g. can't just do `filter(!is.na(df_col))`)

- I really miss defaultdict from python, and dictionaries generally.

- Passing variable names as strings to dynamically generate things seems clunky compared with python. Again, it may be because I'm doing to wrong but I end up having to wrap things in !!sym the whole time and the nse semantics seem hard to remember (I only use R about 20% of the time). I liked cur_data() for passing a df row to a function but this now seems deprecated.

- String formatting -- fstrings are just great. Glue is OK, but escaping special characters seems more tricksy. Jinjar is OK, not quite jinja.

- purrr is nice, but furrr just isn't a drop-in replacement. Making http requests in parallel seems non-trivial compared to doing it with python. Is there an easy way to do it without creating multiple processes? Why can't I just do something like `. %>% mutate_parallel(response=GET(url), workers=10) %>% ...`?

Re: R: Introduction to Data Science (2019)

#43

Post by ekianjo »

mushufasa wrote: post #39
hadley wrote: post #20

Earlier quoted context omitted.

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

A few of the main issues I see, as a R user who built his company on python - when we wanted to build a web app that processes data, it was a lot more straightforward to build both in python, so we can process data within the web servers instead of having to manage multiple stages of infrastructure and different languages. There's no Django for R. - R will often do something instead of explicitly failing. This is the…

> one of a hundred packages was updated

There's renv that addresses that point already: https://rstudio.github.io/renv/articles/renv.html

> There's no Django for R.

Nowadays you can integrate R with WebR (WASM) in a web app: https://docs.r-wasm.org/webr/latest/

Re: R: Introduction to Data Science (2019)

#44

Post by doodledoodahs »

mushufasa wrote: post #39
hadley wrote: post #20

Earlier quoted context omitted.

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

A few of the main issues I see, as a R user who built his company on python - when we wanted to build a web app that processes data, it was a lot more straightforward to build both in python, so we can process data within the web servers instead of having to manage multiple stages of infrastructure and different languages. There's no Django for R. - R will often do something instead of explicitly failing. This is the…

> R will often do something instead of explicitly failing.

I mentioned exception handling above, but this is more specifically the problem.

I think it's a hard problem to solve, because the behaviour of older libraries is so varied.

I have sometimes thought that something like a try catch wrapper which pattern matched or tested the value returned would be useful.

Re: R: Introduction to Data Science (2019)

#45

Post by laylower »

benreesman wrote: post #40

I'm looking at R seriously for the first time. I've got a decade in with Python numeric computing, and I'm interested in Julia and all of the cutting-edge stuff. I've only dabbled with R until now, and I haven't researched it enough to know if rumors of it's inevitable demise have any substance. There are a lot of interesting math problems other than training gigantic neural networks on NVIDIA gear, and I've got some…

I love R. You could do it R. But a lot of the derivations and Math Finance stuff you can and should be able to do in C/C++. R packages mostly depend on those as well for heavy duty calcs.

So, if I wanted to dabble I'd easily use R and if I was in the quant developer world I'd be doing C/C++

Re: R: Introduction to Data Science (2019)

#46

Post by ekianjo »

vharuck wrote: post #41
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

Why I still use R for analysis at work: - R Markdown is just great for static reports. We use PowerBI or ArcGIS for interactive stuff. - GIS is a breeze. My work provides licenses for ArcGIS, which has a Python library for scripting. Despite that, it is so much easier to do stuff in R, which can read and create ArcGIS shapefiles. - Exploratory data analysis is easy. Often, before meetings, I'll connect to the databas…

> R Markdown is just great for static reports.

Quarto (also supporting R) is a good replacement for rmarkdown (with a saner syntax) and I say this as someone who has extensively used rmarkdown over the years.

Re: R: Introduction to Data Science (2019)

#47

Post by doodledoodahs »

usgroup wrote: post #10
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

I think its more intuitive for statistical applications where Python is grossly under-represented. This includes things like the design and analysis of experiments but also lots of domain specific statistics and algorithms such as in bioinformatics, chemistry, and so on. Typically those applications are not the sort of line-of-business enhancements ML in Python is more tuned to. I.e. recommender systems, NN models, a…

The consistency of model specification across multiple libraries is really helpful (base lm, lme4, brms etc). Even though the syntax is sometimes extended, it seems consistent enough to mostly be comprehensible/guessable.

Re: R: Introduction to Data Science (2019)

#48

Post by laylower »

doodledoodahs wrote: post #42
hadley wrote: post #20

Earlier quoted context omitted.

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

OK, since you're here! (this all prefaced with a massive thank you for tidyverse, without which R is very crusty). I love R for interactive work and quick analyses, but I'm currently trying to integrate various bits of R code into a large document-building pipeline and wishing I could use Python for it: - Exception handling and error processing seem a pain in R. Maybe I'm doing it wrong, but if feels like a mess and…

Amen to that. Can I add the following:

- 5 different ways to do wide to long and long to wide over the years even in the tidyverse. - A lot of dependencies to connect to DBs and difficult programs. Rstudio/Posit does have some premium libraries but they should be made free and bundled with the tidyverse to really promote the ecosystem. - Shiny support to save interactive charts and tables. This is a massive problem for me. If I have a heavily stylized HTML table with a bunch of css, I need to rely on webshot, webshot2 which are both alpha or beta versions and they are poorly documented. How can I evangelize R if my deployments cannot be used properly by my community?

Re: R: Introduction to Data Science (2019)

#49

Post by helsinki »

laylower wrote: post #45
benreesman wrote: post #40

I'm looking at R seriously for the first time. I've got a decade in with Python numeric computing, and I'm interested in Julia and all of the cutting-edge stuff. I've only dabbled with R until now, and I haven't researched it enough to know if rumors of it's inevitable demise have any substance. There are a lot of interesting math problems other than training gigantic neural networks on NVIDIA gear, and I've got some…

I love R. You could do it R. But a lot of the derivations and Math Finance stuff you can and should be able to do in C/C++. R packages mostly depend on those as well for heavy duty calcs. So, if I wanted to dabble I'd easily use R and if I was in the quant developer world I'd be doing C/C++

I work with a trading team that manages $1B, exclusively with R.

Re: R: Introduction to Data Science (2019)

#50

Post by lottin »

chollida1 wrote: post #12
ildjarn wrote: post #6

Earlier quoted context omitted.

R is much better for REPL style development and functional programming. Python could be so much better with some minor syntax extensions.

I find that with vscode and the immediate window I get a decent repl. What about R's language makes it better for Repl driven development?

For one thing, R code can be written more concisely, due to the fact that the language is vector-based and functionally-oriented.
Post reply on HN