Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An opinionated view of the Tidyverse “dialect” of the R language

github.com

Post reply on HN
Print view View original

21–30 of 156 posts

Re: An opinionated view of the Tidyverse “dialect” of the R language

#21

Post by kickout »

educationdata wrote: post #15
throw20102010 wrote: post #13

I think he’s exaggerating the effect of some of these claims. People who “grew up” learning R+Tidyverse will discriminate against non-Tidyverse users? Give me a break. You are either an advanced or competent or novice R user. You either understand how pipes (%>%) work or you don’t. I doubt that anyone is going to be denied a job because they are an amazing R programmer but they just don’t have the experience with a p…

"cherry picked"? There is no doubt that data.table is generally way faster than dplyr, unless you cherry pick a few use cases. The problem is that dplyr is much slower than data.table and RStudio is promoting tidyverse too much to make the slower choice a default for many users. The article is 100% correct on this issue.

See my other comment, but I would be curious to know how many users actually could detect the speed difference between a data.table and a tidy solution. Speaks to how small most datasets really are IMO

Re: An opinionated view of the Tidyverse “dialect” of the R language

#22

Post by atheriel »

Bootvis wrote: post #5
kickout wrote: post #3

Decent critiques in my opinion. I was trained on/learned base-R myself ~10 years ago and love the "tidyverse" (although the author is correct, data.table is superior for big data and its not particularly close). Can't imagine people that only know tidyr/dplyr/etc without knowing base-R, seems like those people would get exposed quickly. I enjoy base-R solutions where I can. I use data.table when my data sets are medi…

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

Absolutely. I usually respond to the "R-is-not-a-real-language" line with something like "R is syntactic sugar on a Lisp where the atoms are APL arrays". It's plenty interesting from a computer science perspective, if you bother to look.

For instance, why does R use <- instead of = for assignment? Because initial versions (of S) predate C -- developed down the hall -- the language that first introduced = for assignment.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#23

Post by throw20102010 »

educationdata wrote: post #15
throw20102010 wrote: post #13

I think he’s exaggerating the effect of some of these claims. People who “grew up” learning R+Tidyverse will discriminate against non-Tidyverse users? Give me a break. You are either an advanced or competent or novice R user. You either understand how pipes (%>%) work or you don’t. I doubt that anyone is going to be denied a job because they are an amazing R programmer but they just don’t have the experience with a p…

"cherry picked"? There is no doubt that data.table is generally way faster than dplyr, unless you cherry pick a few use cases. The problem is that dplyr is much slower than data.table and RStudio is promoting tidyverse too much to make the slower choice a default for many users. The article is 100% correct on this issue.

It’s right on this issue, but the Tidyverse is a collection of packages, and most are speedy. Discussing the rare case that supports an argument but neglecting the numerous others that do not support it is the definition of cherry picking.

And I don’t expect RStudio to say, “we have this collection, which works fine for most users, except in this case you should replace package x with y, or in this corner case you might like package z.”

RStudio doesn’t need to promote a fragmented ecosystem if they don’t want to, it won’t cause the death of R.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#24

Post by vasili111 »

dracodoc wrote: post #8

One reason dplyr was promoted is that you can connect to different server backend like sparkr, sql etc, which is the enterprise direction RStudio aiming at. In the other hand, data.table is your friend when you are using your own machine. With more and more memory and cpu power available, its benefits are actually increasing. I have been using data.table from almost day one, it does worth more recognition. The syntax…

If my university uses Enterprise RStutio on grid it is better to stick with tidyverse rather than data.table?

Re: An opinionated view of the Tidyverse “dialect” of the R language

#25

Post by nerdponx »

Bootvis wrote: post #5
kickout wrote: post #3

Decent critiques in my opinion. I was trained on/learned base-R myself ~10 years ago and love the "tidyverse" (although the author is correct, data.table is superior for big data and its not particularly close). Can't imagine people that only know tidyr/dplyr/etc without knowing base-R, seems like those people would get exposed quickly. I enjoy base-R solutions where I can. I use data.table when my data sets are medi…

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

The R standard library is kind of weird and hard to use for "general purpose" programming. It's very clearly a domain specific language.

Also, it's unbelievably slow for basic operations like looping, function calls, and variable assignment. It's literally orders of magnitude slower than Python (I've tested it). Unlike its fellow C-flavored-Lisp Javascript, R retains an extreme level of homoiconicity, Which apparently makes the language very difficult to compile or optimize. I don't know how e.g. SBCL does it, but over the lifetime of the R language no one has managed to implement a non-trivial subset of R that performs better than GNU R (fastR was never finished to my knowledge). So you are basically relegated to writing code in C, Fortran, or C++ if you need even decent performance. Otherwise you are stuck using "vectorized" operations like lapply(), which are fine but make for a jarring experience if you're coming from other languages.

So of course it's a "real" language in the sense that Bash is a real language. But it's ultimately and fundamentally a domain-specific scripting language and I don't know if there is a way around that.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#26

Post by atheriel »

kickout wrote: post #3

Decent critiques in my opinion. I was trained on/learned base-R myself ~10 years ago and love the "tidyverse" (although the author is correct, data.table is superior for big data and its not particularly close). Can't imagine people that only know tidyr/dplyr/etc without knowing base-R, seems like those people would get exposed quickly. I enjoy base-R solutions where I can. I use data.table when my data sets are medi…

I'm in a similar boat, using base R, data.table, or tidyverse as needed. But over time I've found that I respect, more and more, the good decisions that have been made in base R.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#27

Post by throw20102010 »

kickout wrote: post #17
throw20102010 wrote: post #13

I think he’s exaggerating the effect of some of these claims. People who “grew up” learning R+Tidyverse will discriminate against non-Tidyverse users? Give me a break. You are either an advanced or competent or novice R user. You either understand how pipes (%>%) work or you don’t. I doubt that anyone is going to be denied a job because they are an amazing R programmer but they just don’t have the experience with a p…

He was careful how he said it; thus the many disclaimers about being a fan of tidyverse in general. I think you are underestimating the effect of tidyverse being taught. Also, RStudio is absolutely free to push whatever packages they want. Same as R users in general are free to use any package they want. But if most people learn the tidyverse, thats invariably what they will default to whether people like it or not.

Sure, he puts the required disclaimers about respect in the beginning so that way he doesn't get accused of being hateful. But then he accuses RStudio of doing "an end run around the core R leadership team," and that what they are doing is "bad for the health of the project," which are pretty strong words.

RStudio isn't on the leadership team. If the language ends up dying (which it won't, R is dug into it's place like a tick), it would be due to the leadership team's refusal to adapt to good ideas coming in from the community.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#28

Post by pgcudahy »

In this article and others I've read, people complain about the tidyverse's lack of performance, but I think that places too much emphasis on speed of execution versus speed of development. As an academic, most of the R users I know only code as a portion of their scientific projects. Besides data analysis we're doing data collection, manuscript writing and grant writing. The tidyverse's more english-like syntax (eg select() versus `[`) and following a series of pipes rather than unnesting ten sets of brackets makes it so much easier to come back to my code after a few weeks or months and pick up where I left off, or to work with other people's code. My time is more valuable than computer time so the tidyverse is my choice.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#29

Post by cwyers »

educationdata wrote: post #19
cwyers wrote: post #16

Earlier quoted context omitted.

Depending on the size of your data, you might not care that dplyr is slower than data.table. If you're better at writing/composing dplyr, you can often make up the speed difference between the two in terms of the savings in time spent writing and reading code. And if your data is that large, there are solutions like dbplyr out there to run dplyr code on various backends and offload the computation outside of R.

> If you're better at writing/composing dplyr, you can often make up the speed difference between the two in terms of the savings in time spent writing and reading code. dplyr syntax is definitely more concise and readable than base R, but comparing to data.table I don't think it has any advantage in terms of saving time writing or reading code.

I think the article sort of punts on providing examples of a complicated set of operations on a data frame. dplyr's author provides what I think is a good example of the differences between data.table and dplyr on a reasonably complex problem:

https://stackoverflow.com/questions/21435339/data-table-vs-d...

I feel like the first example is far more readable than the second. People can disagree on this, but the adoption rates of dplyr versus data.table do suggest (don't prove, but suggest) that the consensus on the issue leans towards dplyr. As we've noted, people certainly aren't adopting dplyr for the speed.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#30

Post by sghi »

As someone who uses and teaches R extensively (and loves the tidyverse) the tidyverse is so much easier to teach to people without any programming experience and who have very little faith in their tech or maths skills (which was me as well).

The tidyverse just 'made sense' to me when I started using R for the first time a few years ago, and now I love using R and programming. On the other hand, some of my ex-classmates learnt base R (because that's what we were taught) and found it hard, didn't learn anything properly, and now still think R or other programming languages are opaque and hard.

I'm not particularly fussed if Statistics Profs prefer data.table to dplyr or base R to tidyr, I know what is easier to teach, understand and use for me and a lot of other ecology/bio students and people.

Post reply on HN