Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big Book of R

bigbookofr.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: Big Book of R

#61

Post by wodenokoto »

wwweston wrote: post #40
cye131 wrote: post #37

R especially dplyr/tidyverse is so underrated. Working in ML engineering, I see a lot of my coworkers suffering through pandas (or occasionally polars or even base Python without dataframes) to do basic analytics or debugging, it takes eons and gets complex so quickly that only the most rudimentary checks get done. Anyone working in data-adjacent engineering work would benefit from R/dplyr in their toolkit.

what’s the story integrating R code into larger software systems (say, a saas product)? I’m sure part of Python’s success is sheer mindshare momentum from being a common computing denominator, but I’d guess the integration story is part of the margins. Your back end may well already be in python or have interop, reducing stack investment and systems tax.

It's getting a lot better, but R in production was something companies 10 years ago would say "so we figured out a way".

The problem is pinning dependencies. So while an R analysis written using base R 20 or 30 years ago works fine, something using dplyr is probably really difficult to get up and running.

At my old work we took a copy of CRAN when we started a new project and added dependencies from then.

So instead of asking for dplyr version x.y, as you'd do ... anywhere, we added dplyr as it and its dependencies where stored on CRAN on this specific date.

We also did a lot of systems programming in R, which I thought of as weird, but for the exact same reason as you are saying for Python.

But R is really easy to install, so I don't see why you can't setup a step in your pipeline that does R - or even both R and Python. They can read dataframes from eachothers memory.

Re: Big Book of R

#62

Post by dkga »

uptownfunk wrote: post #30

I will say, now after 15 years messing with this. With LLM I just do it all in Python. But, I still miss the elegance and simplicity of R for data manipulation and analysis. Especially the dplyr semantics. They really nailed it. I think they got crushed by the namespace / import system. There’s something about R that makes you so fluid and intuitive. But the engineering, the efficiency, I get with Python now, I can’t…

I agree with all your comment… except the very last bit. Do you really find python to be more efficient at engineering stuff than R? And especially speed, which in my experience at least is broadly the same if not faster with R because it interages easier with Rust and C++?

Re: Big Book of R

#63

Post by vhhn »

wwweston wrote: post #40
cye131 wrote: post #37

R especially dplyr/tidyverse is so underrated. Working in ML engineering, I see a lot of my coworkers suffering through pandas (or occasionally polars or even base Python without dataframes) to do basic analytics or debugging, it takes eons and gets complex so quickly that only the most rudimentary checks get done. Anyone working in data-adjacent engineering work would benefit from R/dplyr in their toolkit.

what’s the story integrating R code into larger software systems (say, a saas product)? I’m sure part of Python’s success is sheer mindshare momentum from being a common computing denominator, but I’d guess the integration story is part of the margins. Your back end may well already be in python or have interop, reducing stack investment and systems tax.

There are so many options to emded R in any kind of system. Thanks to the C API, there are connectors for any if the traditional language. There is also RServe and plumber for inter-process interaction. Managing dependencies is also super easy.

My employer is using R to crunch numbers enbeded in a large system based on microservices.

The only thing to keep in mind is that most people writing R are not programmers by trade so it is good to have one person on the project who can refactor their code from time to time.

Re: Big Book of R

#64

Post by pletnes »

shoemakersteve wrote: post #48
dajtxx wrote: post #46

Earlier quoted context omitted.

I am working on a system at present where the data scientist has done the calculations in an R script. We agreed upon an input data.frame and an output csv as our 'interface'. I added the SQL query to the top of the R script to generate the input data.frame and my Python code reads the output CSV to do subsequent processing and storage into Django models. I use a subprocess running Rscript to run the script. It's not…

Any reason you're using CSV instead of parquet?

Many of the reasons csv is bad is because you don’t control both reader and writer. Here, if you’re 2 persons that collaborate OK, they should be fine.

Re: Big Book of R

#65

Post by account-5 »

DadBase wrote: post #13
wpollock wrote: post #3

Very nice, but instead of an owl, shouldn't the cover illustration be a pirate?

Totally agree. R is pure pirate energy. Half the functions are hidden on purpose, the other half only work if you chant the right incantation while facing the CRAN mirror at dawn.

I've never used R before, why would functions be hidden on purpose? Sounds like a recipe for frustration.

Re: Big Book of R

#67

Post by disgruntledphd2 »

JackeJR wrote: post #39
kasperset wrote: post #38

Earlier quoted context omitted.

I love R and dplyr. It is very readable and easy to explain to non-programmers. I use it almost everyday. Not exactly on the topic,I am having difficulties debugging it. May be I need to brush up on debugging R. Not sure if there is a easy way to add breakpoint when using vscode.

browser() ?

trace subsumes browser, it's much more flexible and can be applied to library code without editing it.

Re: Big Book of R

#68

Post by oscarbaruffa »

I'm the curator of Big Book of R and am really happy to see it on the front page of HN :). New books are added every 6 weeks or so and I send a notifications of the new adds to my newsletter subs. Link is at the footer of every page

Re: Big Book of R

#69

Post by thangalin »

juujian wrote: post #8
thangalin wrote: post #7

Tangentially, R can help produce living Markdown documents (.Rmd files). A couple of ways include pandoc with knitr[0] or my FOSS text editor, KeenWrite[1]. I've kept the R syntax in KeenWrite compatible with knitr. Living documents as part of a build process can produce PDFs that are always up-to-date with respect to external data sources[2], which includes source code. [0]: https://yihui.org/knitr/ [1]: https://kee…

Last time I was working on something complex, I was able to knit from Rmd to md, and then use my usual pandoc defaults, which was quite neat. Big recommendation on that workflow.

My typesetting Markdown series explores weaving knitr and pandoc together:

https://dave.autonoma.ca/blog/2019/07/11/typesetting-markdow...

However, most workflows and nearly all editors don't support interpolated variables. To address this, first I developed a YAML preprocessor:

https://repo.autonoma.ca/yamlp.git

Then I grew tired of editing YAML files, piping files together, and maintaining bash scripts. So next, I developed KeenWrite to allow use of interpolated variables directly within documents from a single program. The screenshots show how it works:

https://keenwrite.com/screenshots.html

Re: Big Book of R

#70

Post by itsmevictor »

kasperset wrote: post #38
cye131 wrote: post #37

R especially dplyr/tidyverse is so underrated. Working in ML engineering, I see a lot of my coworkers suffering through pandas (or occasionally polars or even base Python without dataframes) to do basic analytics or debugging, it takes eons and gets complex so quickly that only the most rudimentary checks get done. Anyone working in data-adjacent engineering work would benefit from R/dplyr in their toolkit.

I love R and dplyr. It is very readable and easy to explain to non-programmers. I use it almost everyday. Not exactly on the topic,I am having difficulties debugging it. May be I need to brush up on debugging R. Not sure if there is a easy way to add breakpoint when using vscode.

Have you checked this extension? https://marketplace.visualstudio.com/items?itemName=RDebugge...
Post reply on HN