Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R: Introduction to Data Science (2019)

rafalab.dfci.harvard.edu

Post reply on HN
Print view View original

71–80 of 139 posts

Re: R: Introduction to Data Science (2019)

#71

Post by hadley »

civilized wrote: post #23
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

> We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals In my org we have several 100% R teams (including mine) that have been developing and maintaining business-critical, data-intensive applications for a decade now. We don't find R difficult to integrate into data pipelines. We write our data pipelines in R, and we find it very efficient to do so. T…

If you have specific issues around error messages and tracebacks please feel free to let me know directly or to file issues on Github. We really do care about the legibility of errors and tracebacks and me and my team have put a lot of effort into them in the last few years. But there's always room to do better and I'd love to know where the pain points are.

(The intersection of tidyverse and shiny tracbacks are a known pain point that's hard to resolve. Unfortunately shiny and tidyverse did a bunch of parallel work that took us in slightly different directions and now it's hard to re-align.)

One thing we are missing is a guide to reading traceback for newer users. Often experts can get a good sense of where the problem is, but we've failed to teach newer users how to get the most value from a traceback.

Re: R: Introduction to Data Science (2019)

#72

Post by hadley »

disgruntledphd2 wrote: post #52
tropical333 wrote: post #34

Earlier quoted context omitted.

> What we found was that R had alot of packages but most haven't been touched in years and when you contact the owner you find they've often moved onto the python/pandas/scikit eco system As a "bilingual" R & Python user, I've found this to be true for the latter language as well :) I don't have much to add on top of what other useRs have mentioned, except another testimonial that our company has successfully used R…

Plotnine is a pretty rocking ggplot clone in Python. Just import star and you're golden.

We (Posit) have hired Hassan (the maintainer of plotnine) so this is great to hear :)

Re: R: Introduction to Data Science (2019)

#73

Post by hadley »

klmr wrote: post #30
zhdc1 wrote: post #16

Earlier quoted context omitted.

I’ve transitioned a lot of my work over to Julia, but R is still the most intuitive language I’ve used for scripting out data collection, cleaning, aggregation, and analysis cases. The ecosystem is simply better. The folks who maintain CRAN do a fantastic job. I can’t remember the last time a library incompatibility led to a show stopper. This is a weekly occurrence in Python.

> I can’t remember the last time a library incompatibility led to a show stopper. Oh, it’s very common unless you basically only use much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.

I'd love to hear more about this because from my perspective renv does seem to solve 95% of the challenges the folks face in practice. I wonder what makes your situation different? What are we missing in renv?

Re: R: Introduction to Data Science (2019)

#74

Post by hadley »

wodenokoto wrote: post #37
klmr wrote: post #30

Earlier quoted context omitted.

> I can’t remember the last time a library incompatibility led to a show stopper. Oh, it’s very common unless you basically only use much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.

At my old job we snapshotted CRAN and pinned versions of package dependencies _against_ CRAN.

We now provide snapshotted CRAN binaries (for many platforms) at https://packagemanager.posit.co.

Re: R: Introduction to Data Science (2019)

#75

Post by nequo »

iainctduncan wrote: post #57
laylower wrote: post #45

Earlier quoted context omitted.

I love R. You could do it R. But a lot of the derivations and Math Finance stuff you can and should be able to do in C/C++. R packages mostly depend on those as well for heavy duty calcs. So, if I wanted to dabble I'd easily use R and if I was in the quant developer world I'd be doing C/C++

I've done some work for scientists where they used C++ extensions to R for heavy number crunching. For their workflow, R is really nice. Don't know how common this is though.

Rcpp is pretty common in major performance sensitive packages. The CppCast did an interview with Dirk Eddelbuettel about it in 2022:

https://cppcast.com/rcpp/

Re: R: Introduction to Data Science (2019)

#76

Post by axpy906 »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

I stopped using it in 2015, when I began to learn how to code.

At my FAANG company, there are teams that use it for econometrics. I think that’s Rs sweet spot, still in 2024.

Re: R: Introduction to Data Science (2019)

#77

Post by hadley »

SoftTalker wrote: post #61

I took a two or three day on-site intro to R class that my employer put together. Perhaps it was not a great class, but as a seasoned software developer familiar with a number of imperative and functional languages I was baffled by R. It felt like a bunch of little functions that had been developed by different people with no consistent framework, and thrown together in some kind of big wrapper. I know it's popular a…

R definitely has its warts, but I strongly believe that underneath them lies a beautiful and quite elegant language that's extremely well suited to the challenges of data analysis. If you're already a programmer, you might find something like Advanced R (https://adv-r.hadley.nz) to be useful to get a sense of what R really is as a programming language.

Re: R: Introduction to Data Science (2019)

#78

Post by laylower »

hadley wrote: post #70
laylower wrote: post #48

Earlier quoted context omitted.

Amen to that. Can I add the following: - 5 different ways to do wide to long and long to wide over the years even in the tidyverse. - A lot of dependencies to connect to DBs and difficult programs. Rstudio/Posit does have some premium libraries but they should be made free and bundled with the tidyverse to really promote the ecosystem. - Shiny support to save interactive charts and tables. This is a massive problem f…

What are the premium packages you're talking about? As far as I know all of our R packages are 100% open source. I'd love to hear more why you're using webshot etc to talk screenshots of your shiny app. A more typical workflow would be to generate a separate HTML/PDF with quarto/RMarkdown.

Thanks for responding and your amazing work with the tidyverse. I am the "R-guy" in my finservices company and we have a paid rconnect dev/qa/prod and rserver pro licences for a few hundred users.

The packages I think are the dependencies of some DB connectivity libraries. https://www.rstudio.com/tags/databases/ - these are the ones I was referring to.

Re webshot my use case is: I have a heavily modified DT table in a shiny app. Users log in, play around with the DT table, update ggplots etc and then download the snapshot and send it to a WORD file. I can't move away from word and use html or pdf because we need the word file formatted by editors for publication and they need to follow the corpo guidelines. So, I am having to use webshot to grab a screenshot of the tagged html instead of natively handling it. I tried using officedown and a few other methods and it just didn't work.

ps: I hope the rebrand goes great and I am rooting for you.

Re: R: Introduction to Data Science (2019)

#80

Post by xapata »

civilized wrote: post #23
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

> We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals In my org we have several 100% R teams (including mine) that have been developing and maintaining business-critical, data-intensive applications for a decade now. We don't find R difficult to integrate into data pipelines. We write our data pipelines in R, and we find it very efficient to do so. T…

Why Polars and not Dask?
Post reply on HN