Live data from Hacker News

D for Data Science: Calling R from D

dlang.org

11–20 of 23 posts

Re: D for Data Science: Calling R from D

#11
post #5

Earlier quoted context omitted.

My guess would be leveraging all the libraries and packages in R, without having to rewrite them in D.

Here's what I'm not getting: Most the tasks that require heavy computation in R are done through C/C++/Fortran APIs - why can't D interface with them without the intermediate R layer? The ones that are R-native - visualisation/dataframes - are best done through R itself - what's the use case for doing that through D at all?

I can think of a few different use cases:

1) Use a library written in raw R that is difficult to implement in another language. It can be time-consuming to replace the work done in raw R.

2) Use a library that relies on a C/C++/Fortran API, but then provides additional error checking or providing additional calculations using raw R on top of it. It can be time-consuming to replace the work done in raw R.

3) Use a library where the R implementation has an ecosystem built around it. It can be time-consuming to replace the R ecosystem.

4) One way that I used this in the past was with Stan. Stan is written in C++, but they do not provide a C++ API and you either call it at the command line with cmdstan or use an interface like rstan or pystan. The problem with calling it at the command line is that you have to write the data to a hard disk first. You don't have to do that with rstan. My recollection was that it was faster to call rstan from D than using cmdstan for large data sets. So basically, I can process all the data in D, which tends to be faster than R or Python, and then pass it off to Stan.

5) When attempting to produce a D library with similar features as an R package, one can create a unit test that calls the R version and check that they provide the same results.

Re: D for Data Science: Calling R from D

#12
post #5

Earlier quoted context omitted.

My guess would be leveraging all the libraries and packages in R, without having to rewrite them in D.

Here's what I'm not getting: Most the tasks that require heavy computation in R are done through C/C++/Fortran APIs - why can't D interface with them without the intermediate R layer? The ones that are R-native - visualisation/dataframes - are best done through R itself - what's the use case for doing that through D at all?

The intermediate R layer is often a well thought out abstraction - hence it makes a lot of sense to re-use it. At different points in my career I have written skunkworks wrappers for rpart (decision tree learner) and glmnet (generalized linear models with elasticnet). While its true that some subset of the features of these libraries exist in other languages (my primary working language is Python), these are not as feature-rich. To consider the first example, rpart offers the concept of "surrogate splits"[1] that lacks in scikit decision trees. Also scikit doesn't support categorical features (you need to encode them into one-hot vectors), and rpart does.

In short, the intermediate layers often give you well thought out features.

In some cases, you might not even have a corresponding library in your language. For ex, if you wanted to use interaction terms in your linear model, that respects hierarchies, there aren't many options around, but R has glinternet [2].

[1] https://stats.stackexchange.com/questions/50310/how-does-rpa...

[2] https://cran.r-project.org/web/packages/glinternet/index.htm...

Re: D for Data Science: Calling R from D

#13
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

We’d love to call R from php or python. Incredibly good stats. We use the Rscript (To run r from command line) to generate graphs for the webs but to call directly would be nice.

We tried extracting some of the phyper code into a stand alone application, but not trivial.

Re: D for Data Science: Calling R from D

#14
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

It's really nice to call ggplot2 from Haskell with inline-r quasiquotation, which is evaluated using an instance of the R interpreter embedded in the binary https://tweag.github.io/HaskellR/

Re: D for Data Science: Calling R from D

#15
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

We’d love to call R from php or python. Incredibly good stats. We use the Rscript (To run r from command line) to generate graphs for the webs but to call directly would be nice. We tried extracting some of the phyper code into a stand alone application, but not trivial.

Rpy2 should work, no? And reticulate is really great for calling python from R https://rstudio.github.io/reticulate/index.html

Re: D for Data Science: Calling R from D

#16
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

I'm the author of the blog post. That's actually the interop I normally use, writing D functions and then calling them from R. D calling R is for someone wanting to mostly write D code but is missing a few statistical libraries.

Re: D for Data Science: Calling R from D

#17
post #5

Earlier quoted context omitted.

My guess would be leveraging all the libraries and packages in R, without having to rewrite them in D.

Here's what I'm not getting: Most the tasks that require heavy computation in R are done through C/C++/Fortran APIs - why can't D interface with them without the intermediate R layer? The ones that are R-native - visualisation/dataframes - are best done through R itself - what's the use case for doing that through D at all?

> Most the tasks that require heavy computation in R are done through C/C++/Fortran APIs - why can't D interface with them without the intermediate R layer?

You absolutely can do that, but it's generally not a fun experience. Most of the time the overhead of calling into R is trivial. In that case, take advantage of the convenience of R and get on to other things. There's also a lot of pure R code that you don't have any other way to call.

Re: D for Data Science: Calling R from D

#18
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

We’d love to call R from php or python. Incredibly good stats. We use the Rscript (To run r from command line) to generate graphs for the webs but to call directly would be nice. We tried extracting some of the phyper code into a stand alone application, but not trivial.

https://rpy2.readthedocs.io/en/latest/

Re: D for Data Science: Calling R from D

#19
post #6
post #5

Earlier quoted context omitted.

Here's what I'm not getting: Most the tasks that require heavy computation in R are done through C/C++/Fortran APIs - why can't D interface with them without the intermediate R layer? The ones that are R-native - visualisation/dataframes - are best done through R itself - what's the use case for doing that through D at all?

The matrix algebra and array manipulation is written in C++. The statistical analysis is done in R, potentially using multiple different matrix algebra routines. For example. I have no idea why you would not use R (or python) «at the top», though. R might not have great libraries for network protocols (REST, etc), as it’s not a general language, but more stats oriented.

If you're doing a simulation, for instance, and you're missing a piece of functionality available in R, you can bring in just that piece. D's a nice language, and if you decide you want to use it, it's great to know that every line of R code you've ever written and every R library you've ever called is still available.

Re: D for Data Science: Calling R from D

#20
post #14
post #2

Does anyone actually need to do this? Doesn't interoping in the other direction (calling D from R) at least make a bit more sense?

It's really nice to call ggplot2 from Haskell with inline-r quasiquotation, which is evaluated using an instance of the R interpreter embedded in the binary https://tweag.github.io/HaskellR/

D has a similar ggplot package: https://code.dlang.org/packages/ggplotd
Post reply on HN