I find R's strengths lie in its unmatched collection of statistical libraries, but I dislike R's syntax so much that, if forced to use it, would call an R package from Python (using RPy2), or just use a Python alternative (e.g. Plotnine).
One Year with R
251–260 of 266 posts
Re: One Year with R
#252Earlier quoted context omitted.
I mean that's fair, but if you have to write an academic paper, your options are more or less use a notebook, or copy and paste your results. Of course, the question is how much code you should write in the notebook, versus having it in a more organized set of functions and libraries. It's very easy to end up with a huge bloated document which contains thousands of lines of spaghetti.
I absolutely agree with this: > the question is how much code you should write in the notebook, versus having it in a more organized set of functions and libraries. It's very easy to end up with a huge bloated document which contains thousands of lines of spaghetti. But this isn't correct: > your options are more or less use a notebook, or copy and paste your results. What's wrong with writing scripts that write imag…
We might be talking at cross purposes. I'm thinking of e.g. a Rmarkdown notebook, which is indeed a script not (necessarily) an interactive notebook like Jupyter. And it can be automatically compiled, via a makefile or something similar, and put into version control.
The point is that it makes sense to mix english prose + code to e.g. produce tables or graphs, even if most of the heavy lifting is done separately in code files.
Re: One Year with R
#253Earlier quoted context omitted.
This is just a quick example - I would be grateful if people could recreate this brief look at UK COVID figures in another language: library(tidyverse) library(scales) download.file(url = "https://api.coronavirus.data.gov.uk/v2/data?areaType=overview&metric=covidOccupiedMVBeds&metric=newAdmissions&metric=newCasesBySpecimenDate&metric=newDeaths28DaysByDeathDate&metric=newPeopleReceivingFirstDose&format=csv", destfile…
import pandas as pd import matplotlib.pyplot as plt import matplotlib.dates as mdates import seaborn as sn df = (pd.read_csv("/tmp/overview_2022-03-21.csv") # i just used curl beforehand .assign(date=lambda x: pd.to_datetime(x["date"])) .set_index("date") .melt(value_vars=[ "newCasesBySpecimenDate", "covidOccupiedMVBeds", "newAdmissions", "newDeaths28DaysByDeathDate"], var_name="Data", ignore_index=False) .assign(Dat…
Thanks for providing that. Interesting to see the '.' used like a pipe. Always thought of it used in an OOP context, but interesting that it can be functional too. I also had difficulty fitting a LOESS curve to the plot, but I could do a linear model. The LOESS would have been possible doing it manually I guess.
Re: One Year with R
#254Earlier quoted context omitted.
I absolutely agree with this: > the question is how much code you should write in the notebook, versus having it in a more organized set of functions and libraries. It's very easy to end up with a huge bloated document which contains thousands of lines of spaghetti. But this isn't correct: > your options are more or less use a notebook, or copy and paste your results. What's wrong with writing scripts that write imag…
We might be talking at cross purposes. I'm thinking of e.g. a Rmarkdown notebook, which is indeed a script not (necessarily) an interactive notebook like Jupyter. And it can be automatically compiled, via a makefile or something similar, and put into version control. The point is that it makes sense to mix english prose + code to e.g. produce tables or graphs, even if most of the heavy lifting is done separately in c…
Ah, OK, yes I was talking at cross-purposes to some extent then, thanks. (Jupyter notebooks are hopeless in version control due to the JSON format, but I'm not familiar with Rmarkdown notebooks -- do you get sane diffs?)
Yep, so what you say makes sense. Isn't it sometimes a bit overly prescriptive to assume all collaborators use Rmarkdown? (Perhaps not! I used to work in biology and statistics and R was very ubiquitous.)
Re: One Year with R
#255Earlier quoted context omitted.
A Julia solution with Chain and Gadfly might look something like this, although I've translated the R fairly directly so it might not be very idiomatic. import CSV using Chain: @chain using DataFrames import Downloads using Gadfly using Dates @chain begin Downloads.download( "https://api.coronavirus.data.gov.uk/v2/data?areaType=overview&metric=covidOccupiedMVBeds&metric=newAdmissions&metric=newCasesBySpecimenDate&met…
Thank you for that, good to see there's an elegant Julia solution! The last time I was using 'pipes' with Julia, I think I was using DataFramesMeta. I also really like this interactive gadfly plot - reminds me of Matlab, but better. It's been a little while since using Julia, so I'd forgotten about the pre-compiling thing, but generally this code looks pretty nice and clear.
I used to use DataFramesMeta.jl, but eventually I found that the mini-DSL that DataFrames.jl has created is really powerful and not overly verbose. Now, going back to the Tidyverse's syntax makes me feel a little uneasy, like there's just too much magic going on behind the scenes, even though I used it for years with no problems.
Re: One Year with R
#256Earlier quoted context omitted.
> R feels like a language that was built for people who were using excel, I don't think so. Most people who come to R after years of Excel find it just as alien as you do.
I recall when the pipe operator was first being proposed the argument for it was that it'd enable workflows that felt more like Excel. The implication being that indeed, base R is alien to an Excel user. I also recall my pushback was along the lines of "who on earth would want that". Yeah, it's a good thing I'm not the person coming up with these things :)
> I recall when the pipe operator was first being proposed the argument for it was that it'd enable workflows that felt more like Excel.
I have no idea where you get that impression, most Excel power users I have met take a long time to understand how to use the pipe operator in R.
Re: One Year with R
#257I think this is really interesting. The author certainly isn't an expert, for example `result[which(result But that's just why it's useful - R is great when you are an expert, but becoming an expert takes years. The perspective of new users is really important. (I've been using R almost 20 years, have written several packages, and still feel like an amateur. Indeed, I'd never heard of `**` as an alias for `^` until t…
I think a lot of the problem with comparing R to other languages is that a lot people don't get the problem space that R is working in. Science deals a lot with categorical variables, missing data and high dimensional data, and the 'table' or 'dataframe' is adept at storing and working with this information. Under the hood it's just a load of optimised fortran code working matrices, but the code clearly shows what kinds of data manipulations and transformation you are doing to eek the right information and visualisations out from the dataset.
I see problems when people take an imperative approach to solving numerical problems, and something like Python is better suited to that. Also, R isn't really set up to work with matrices like Matlab/Julia are.
Re: One Year with R
#258The points the author mentions are fair but something feels amiss. I have used R heavily and still use it from time to time and I never use most of the functions mentioned in the post. For instance I have never used switch(). R is for data manipulation. 90% of what I do in R is manipulate dataframes or matrices and then run machinelearningmodel(mydataframe) or ggplot(mydataframe). And for this it is incredibly effici…
Yes, I think if you are using switch() for an analysis in R, you're either using the wrong language, or you're doing R wrong.
Re: One Year with R
#259I have been using R for almost 20 years now. I work on a medium-sized quant team at a large asset manager and we run several $BN off R - we mostly trade equities and vanilla derivatives. Our models are primarily statistical/econometric-based. In aggregate, we probably have about a hundred scheduled jobs associated with a variety of models and on the order of 15 shiny applications to facilitate implementation. We have an internal CRAN-like repo and everything we produce is packaged/versioned with gitlab CI/CD. We have RStudio Server at my firm and half my team uses that for development, the other half, including myself, uses emacs/ess. All of us use RConnect for scheduling & application hosting - it has it's quirks, but it's excellent in a constrained IT environment.
I often chuckle when people complain about R in production and how it isn't a good general purpose programming language, my experience has been the polar opposite. You can write bad code in any language, and R is no exception, but R allows you to write so much less code and R-core is truly exceptional at backwards compatibility. Our approach to R is basically:
- Don't have a lot of dependencies, and when you do have dependencies, make sure they themselves don't have a lot of dependencies. While we do use shiny as mentioned above, our core models are very dependency light and shiny is just a basic front end.
- data.table (which was designed by quants) is a zero-dependency package that is by far the best tabular data manipulation package that has ever been created since the dawn of time. We generally work on an EC2 instance running linux with a ton of memory. In the - Check/coerce argument types and lengths on function input to catch and avoid all the quirky edge cases that drive people nuts - it's so easy!
- I hate OOP and I love that R doesn't encourage it. Mutable state, especially for non-software engineers, is the devil. Don't get me wrong, OOP has its place, but the fact that R encourages functional programming is one of the best things about it. The slight inefficiency this produces is almost never a problem.
- R is not slow at all when used correctly. Additionally, the C API is a joy to use when necessary.
- Stick to the base types: vectors, matrices, lists, environments and data.tables (only exception). The fact that you can name, and then use names to index all of the above is stunningly powerful. The only "objects" we really create are lightweight extensions of lists with an S3 print method.
- We have an internal version of renv/packrat that creates a plain text "dependency file" for projects and we pin package versions in docker containers. RConnect doesn't use docker right now, but they do have a versioning system that works quite well in my experience.
I definitely wouldn't want to build something like a company website in R, but I also wouldn't want to build that in C either. R definitely has it's place a server-side language even outside it's assumed domain of statistics.
Haters gonna hate, but joke is on them.
Re: One Year with R
#260I love R more than any other language I have ever used. Perhaps more than any piece of software I've ever used. All of these points are valid, and yes, it's messy, and if you try to write the same type of code that you would in Python, it will frustrate you. And yet.. it somehow works. It makes data analysis and statistical modelling a pleasure. It somehow gives off a sense of lightness, and makes it easy to investig…
I feel the exact same way! I've used R for the past decade. Once you learn the philosophy behind it, it just works. Yesterday my boss asked me a question about a dataset and I wrote code to analyze it while talking through the problem in real time.