Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An opinionated view of the Tidyverse “dialect” of the R language

github.com

Post reply on HN
Print view View original

71–80 of 156 posts

Re: An opinionated view of the Tidyverse “dialect” of the R language

#71

Post by vincent-toups »

Bootvis wrote: post #5
kickout wrote: post #3

Decent critiques in my opinion. I was trained on/learned base-R myself ~10 years ago and love the "tidyverse" (although the author is correct, data.table is superior for big data and its not particularly close). Can't imagine people that only know tidyr/dplyr/etc without knowing base-R, seems like those people would get exposed quickly. I enjoy base-R solutions where I can. I use data.table when my data sets are medi…

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

"Based on Scheme" really, profoundly, oversells it. R's idea of Scheme is deeply bizarre and profoundly outdated.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#72

Post by thom »

hardboiled wrote: post #62
jrumbut wrote: post #41

Earlier quoted context omitted.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly). I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr. That said it is hard…

Tidyverse code is not write-only; it is designed to be mostly read-only where the level of abstraction is at the level of the domain, which in this case is data frames and data pipelines akin to the same constructs in relational algebra/SQL or data processing (map/reduce). Correct usage requires learning the right abstractions, but fortunately these abstractions are shared across language communities and frameworks.…

It’s very hard to write expressive, readable code that munges some horrible tabular format into another arbitrary tabular format, to be fair. That said it’s true that most R code doesn’t seem like map/reduce, or some other pipeline of transformations. It’s usually more like someone cut and pasted a long, hard REPL session into a notebook and has no intention of ever scrolling back up.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#73

Post by minimaxir »

cwyers wrote: post #16
educationdata wrote: post #15

Earlier quoted context omitted.

"cherry picked"? There is no doubt that data.table is generally way faster than dplyr, unless you cherry pick a few use cases. The problem is that dplyr is much slower than data.table and RStudio is promoting tidyverse too much to make the slower choice a default for many users. The article is 100% correct on this issue.

Depending on the size of your data, you might not care that dplyr is slower than data.table. If you're better at writing/composing dplyr, you can often make up the speed difference between the two in terms of the savings in time spent writing and reading code. And if your data is that large, there are solutions like dbplyr out there to run dplyr code on various backends and offload the computation outside of R.

In practice, if there's ever a case that there's "too much" data such that dplyr starts to hang (e.g. millions of rows, hundreds of columns), you would get better value by setting up a database first with the data. Which you can then query with dbplyr!

Re: An opinionated view of the Tidyverse “dialect” of the R language

#74

Post by stilley2 »

I recently tried to do some stuff in R with tidyverse, and was not a fan. I'm no expert, but the tidyverse's frequent use of nonstandard evaluation drove me crazy. It makes it so much more difficult to write functions encapsulating tidy functions. However, I never see people complain about this, so maybe I'm doing something wrong or not grokking something...

Re: An opinionated view of the Tidyverse “dialect” of the R language

#75

Post by vincent-toups »

nerdponx wrote: post #25
Bootvis wrote: post #5

Earlier quoted context omitted.

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

The R standard library is kind of weird and hard to use for "general purpose" programming. It's very clearly a domain specific language. Also, it's unbelievably slow for basic operations like looping, function calls, and variable assignment. It's literally orders of magnitude slower than Python (I've tested it). Unlike its fellow C-flavored-Lisp Javascript, R retains an extreme level of homoiconicity, Which apparentl…

I can give you some sense of why one can compile SBCL and not R. SBCL and other Lisps these days maintain some kind of conceptual barrier between macroexpansion time and run time, so that you can _stage_ your activity appropriately. Read the code, gather the macro definitions, expand the code (repeat as necessary) until you have some base language without metaprogramming in it.

Then you can optimize and compile. This is particularly true of Scheme which foregoes much of the dynamic quality of Common Lisp in favor of a much more static view of the world. But its still basically true in CL.

R doesn't have macros of that kind. It has a sort of weird laziness based on arguments remaining unevaluated until needed. Metaprogramming in R is typically done by intercepting those "quoted" arguments and then evaluating them in modified contexts (this is exposed to the user more or less by letting them insert scopes into the "stack").

Thus, there is no distinction between macroexpansion and execution time. Hence, its tough to write a compiler, which is basically just a program which identifies invariants ahead of time and pre-computes them (eg, lexical scope is so good for compilers because you can pre-compute variable access). Because all sorts of shenanigans can be got up to by walking quoted expressions and evaluating them in modified contexts, R is hard to compile.

This is, by the way, why the `with` keyword was removed from Javascript. It provided exactly the ability to insert a scope into the stack used to look up variable bindings.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#76

Post by vincent-toups »

thom wrote: post #10
Bootvis wrote: post #5

Earlier quoted context omitted.

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

I don't think there's any shame in R effectively being a good, productive DSL for a lot of stats and viz work, built around fast stuff written in C++. I also think this is largely true of a lot of Python as well, tbh. In the future my hope is that projects like Arrow end up with even more of the workload being taken off R's shoulders.

I'm a data scientist and, frankly, I really strongly prefer R over Python. Its mostly the whitespace in Python, but the language is also slow and crufty as heck. R, if you use it responsibly, feels a lot more like your standard, lexically scoped, dynamically typed language.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#77

Post by vincent-toups »

stilley2 wrote: post #74

I recently tried to do some stuff in R with tidyverse, and was not a fan. I'm no expert, but the tidyverse's frequent use of nonstandard evaluation drove me crazy. It makes it so much more difficult to write functions encapsulating tidy functions. However, I never see people complain about this, so maybe I'm doing something wrong or not grokking something...

I really hate this. And its so unnecessary!

Re: An opinionated view of the Tidyverse “dialect” of the R language

#78

Post by mespe »

sghi wrote: post #30

As someone who uses and teaches R extensively (and loves the tidyverse) the tidyverse is so much easier to teach to people without any programming experience and who have very little faith in their tech or maths skills (which was me as well). The tidyverse just 'made sense' to me when I started using R for the first time a few years ago, and now I love using R and programming. On the other hand, some of my ex-classma…

"I know what is easier to teach, understand and use"

I think this really depends on the end point. If you want to learn to read data into R and do basic manipulations, plotting, modeling, etc., the Tidyverse absolutely has a lower bar to entry. Once you get into writing functions, it gets a little trickier. Knowing some of the base R programming concepts and skills will make you much more efficient. If you start debugging and profiling code, only knowing the Tidyverse becomes a liability because you fundamentally do not understand R's computational model (the Tidyverse does not follow it). Hence, if your end goal is to write and debug functions in R, the steeper learning curve of base R can more than pay off. If not, then the Tidyverse's low bar to entry can be more attractive.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#79

Post by educationdata »

minimaxir wrote: post #73
cwyers wrote: post #16

Earlier quoted context omitted.

Depending on the size of your data, you might not care that dplyr is slower than data.table. If you're better at writing/composing dplyr, you can often make up the speed difference between the two in terms of the savings in time spent writing and reading code. And if your data is that large, there are solutions like dbplyr out there to run dplyr code on various backends and offload the computation outside of R.

In practice, if there's ever a case that there's "too much" data such that dplyr starts to hang (e.g. millions of rows, hundreds of columns), you would get better value by setting up a database first with the data. Which you can then query with dbplyr!

data.table can handle millions of rows easily, as long as the data can fit in the memory.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#80

Post by robust »

mespe wrote: post #52

I teach and consult on R and data science. I had the privilege of learning R from one of R's core developers. My students often ask why I don't use the Tidyverse. The answer is because I don't need to - I can do everything the Tidyverse does and so much more in base R. This article only briefly touches on what I think is the biggest issue with the Tidyverse. The Tidyverse is incredibly limiting. The "tidy" workflow h…

Can you provide an example of a problem that requires multiple tidyverse operations, but could be solved equally well using only lapply?
Post reply on HN