Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R: Introduction to Data Science (2019)

rafalab.dfci.harvard.edu

Post reply on HN
Print view View original

101–110 of 139 posts

Re: R: Introduction to Data Science (2019)

#101

Post by ramblenode »

civilized wrote: post #23
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

> We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals In my org we have several 100% R teams (including mine) that have been developing and maintaining business-critical, data-intensive applications for a decade now. We don't find R difficult to integrate into data pipelines. We write our data pipelines in R, and we find it very efficient to do so. T…

> My fondest hope is that in the coming decades there arises a new, thoughtfully designed language with the Lispy flexibility of R, but also optional type safety and static analysis affordances.

I think many of us saw Julia as the successor to R. Unfortunately, the package ecosystem---one of R's strongest points---still has a long way to go.

Re: R: Introduction to Data Science (2019)

#102

Post by nutshell42 »

acc_297 wrote: post #18
chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

Pk/PD work for pharmaceutical data analysis I didn’t like using R at first but I’ve come to appreciate the speed that comes with months of experience. It’s a language which feels like it has a lot of magical incantations you need to remember - the default namespace is much more crowded. Functions like sapply vs mapply are tricky to reason about from the documentation alone. The values NA vs Null vs integer(0) are all…

> Functions like sapply vs mapply are tricky to reason about from the documentation alone.

Could you please expand on that? It's unclear what you're referring to.

> The values NA vs Null vs integer(0) are all used as standins for real thrown errors and knowing which one to check for after calling a function can be tough.

`checkmate::assert_numeric()` (or similar)

with base R you want isTRUE():

`stopifnot(isTRUE(is.finite(x)))` (or is.na or anything else) will error on empty values.

Re: R: Introduction to Data Science (2019)

#103

Post by melondonkey »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

[dead]

Re: R: Introduction to Data Science (2019)

#104

Post by clatan »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

I'm an old R user forced to mostly use python because that's what the team uses.

R is so much better than python in many areas concerning data pipelines: connecting with external database systems through an unified API, superior data munging utilities, as well as plotting, a more comprehensive (obviously) statistical analysis toolset.

I even find rmarkdown vastly superior to jupyter.

But IMO the best reason to use R rather tha python is that its tools will make you approach the problem as a statistician rather than a programmer.

Re: R: Introduction to Data Science (2019)

#105

Post by civilized »

ramblenode wrote: post #101
civilized wrote: post #23

Earlier quoted context omitted.

> We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals In my org we have several 100% R teams (including mine) that have been developing and maintaining business-critical, data-intensive applications for a decade now. We don't find R difficult to integrate into data pipelines. We write our data pipelines in R, and we find it very efficient to do so. T…

> My fondest hope is that in the coming decades there arises a new, thoughtfully designed language with the Lispy flexibility of R, but also optional type safety and static analysis affordances. I think many of us saw Julia as the successor to R. Unfortunately, the package ecosystem---one of R's strongest points---still has a long way to go.

I was excited about Julia too but it now seems to be a relatively niche HPC language. It's about saving CPU time more than user time.

My sniff test for a successor language to R is whether it can replicate the tidyverse API with 100% fidelity. The API is already optimal for tabular data analysis, especially the dplyr core. It can be thought of as a specification for other languages to implement.

There is a great deal about how R works that is negotiable. But if the language can't implement dplyr to spec, or somehow doesn't "want to", it's not the language for the audience served by the tidyverse.

Re: R: Introduction to Data Science (2019)

#106

Post by nutshell42 »

doodledoodahs wrote: post #42
hadley wrote: post #20

Earlier quoted context omitted.

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

OK, since you're here! (this all prefaced with a massive thank you for tidyverse, without which R is very crusty). I love R for interactive work and quick analyses, but I'm currently trying to integrate various bits of R code into a large document-building pipeline and wishing I could use Python for it: - Exception handling and error processing seem a pain in R. Maybe I'm doing it wrong, but if feels like a mess and…

> The distinction between warning, stop etc seems odd. The option to stop on warnings isn't useful because older packages seem to abuse warnings as messages.

Use suppressWarnings() to silence misbehaving functions or withCallingHandlers() to stop or handle specific conditions.

> Passing variable names as strings to dynamically generate things seems clunky compared with python.

Can you give me an elegant example in Python? Because I don't understand what you want to generate dynamically.

That said, I dislike the tidyverse solution as well. Too much abstraction for not enough benefit over a base solution with substitute()

Re: R: Introduction to Data Science (2019)

#107

Post by bomewish »

fastaguy88 wrote: post #88
hadley wrote: post #20

Earlier quoted context omitted.

If you tell me what makes R hard to integrate into data pipelines I will do my best to fix it :)

(1) The big problem I have is transitioning from RStudio to a pipeline (so I end up not using RStudio). A traditional pipeline is going to be a script with some set of arguments -- parameter values, fitting functions, and data file names, that I put into a shell script and say: my_plot_script.R --plot_col=g_max --output_type=pub_quality data_file1 data_file2 data_file3 It's possible to use optparse/OptionParser() to…

On point two, can’t you just use stopifnot(condition)? Then log it etc?

Re: R: Introduction to Data Science (2019)

#108

Post by dxbydt »

benreesman wrote: post #40

I'm looking at R seriously for the first time. I've got a decade in with Python numeric computing, and I'm interested in Julia and all of the cutting-edge stuff. I've only dabbled with R until now, and I haven't researched it enough to know if rumors of it's inevitable demise have any substance. There are a lot of interesting math problems other than training gigantic neural networks on NVIDIA gear, and I've got some…

quite easy to price derivatives with R. I have a degree in finmath from uchicago, where derivative pricing was taught using Matlab and R. But in the last semester we were told - oh yeah when you go out there into the real world and start working for the banks you can’t use civilized tools like R and Matlab. So you have to take this mandatory class on cpp. There once was a guy named stroustrup and this shit here is called a makefile… after graduation i worked for BofA and yes, the quant world is completely C++. But there are small funds (few billion dollars) that do their own shit in R, Haskell, Q/kdb, others. Very doable in R.

Re: R: Introduction to Data Science (2019)

#109

Post by dxbydt »

chollida1 wrote: post #3

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

>what is your use case?

if you are doing Bayesian stats, fitting hierarchical models, or using Stan in any serious capacity, R/Stan is so much more ergonomic than Pystan. Here’s a long list of pros-cons:

https://discourse.mc-stan.org/t/various-observations-on-rsta...

Re: R: Introduction to Data Science (2019)

#110

Post by mushufasa »

hadley wrote: post #67
ekianjo wrote: post #43

Earlier quoted context omitted.

> one of a hundred packages was updated There's renv that addresses that point already: https://rstudio.github.io/renv/articles/renv.html > There's no Django for R. Nowadays you can integrate R with WebR (WASM) in a web app: https://docs.r-wasm.org/webr/latest/

A lighterweight alternative to renv is to use Posit Public Package Manage ( https://packagemanager.posit.co/ ) with a pinned date. That doesn't help if you're installing packages from a mix of places, but if you're only using CRAN packages it lets you get everything as of a fixed date. And of course on the web side you have shiny ( https://shiny.posit.co ), which now also comes in a python flavour.

shiny is nice for one-off data dashboards and single-purpose mini-apps. I see the python equivalents are like dash/plotly. Shiny is not a full fledged web framework, and isn't a viable replacement for e.g. Django.

Aside -- we tried using dash in our production app and then had to remove it after a month, because these types of frameworks that spit out front-end code are almost never flexible enough to do what you actually need to do in a full app context, and you end up doing more work to fight the framework versus the time-savings from the initial prototype.

Post reply on HN