Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An opinionated view of the Tidyverse “dialect” of the R language

github.com

Post reply on HN
Print view View original

61–70 of 156 posts

Re: An opinionated view of the Tidyverse “dialect” of the R language

#61

Post by FranzFerdiNaN »

mespe wrote: post #52

I teach and consult on R and data science. I had the privilege of learning R from one of R's core developers. My students often ask why I don't use the Tidyverse. The answer is because I don't need to - I can do everything the Tidyverse does and so much more in base R. This article only briefly touches on what I think is the biggest issue with the Tidyverse. The Tidyverse is incredibly limiting. The "tidy" workflow h…

"Meanwhile, none of the truly advanced R users I encounter use the Tidyverse."

This is most likely because truly advanced R users have been using R since far before the Tidyverse existed. A whole new generation of R users is being brought up with the Tidyverse, so im curious to see how the situation will be in 10 years time.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#62

Post by hardboiled »

jrumbut wrote: post #41
thom wrote: post #9

Wow, the idea that the Tidyverse is somehow burdened with an overabundance of computer science or functional programming is just hilarious to this programmer who has had to maintain, debug and optimise other people's R code.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly). I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr. That said it is hard…

Tidyverse code is not write-only; it is designed to be mostly read-only where the level of abstraction is at the level of the domain, which in this case is data frames and data pipelines akin to the same constructs in relational algebra/SQL or data processing (map/reduce).

Correct usage requires learning the right abstractions, but fortunately these abstractions are shared across language communities and frameworks.

If you are doing any data processing work and you do not know about foundational functional concepts ie map/reduce then I would argue the code you write is less readable, overly focused on the idiosyncracies of how to do something rather than how entities and data are related to each other - their logic and structure.

The main optimization is to be correct and expressive of one's intention and purpose. If you need performance use Julia.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#63

Post by kgwgk »

> Ironically, though consistency of interface is a goal, new versions of Tidy libraries are often incompatible with older ones, a very serious problem in the software engineering world.

I was (sadly) amused by a recent comment on HN linking to some tweets celebrating that some R code still ran four years later.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#64

Post by [deleted] »

epistasis wrote: post #33
Bootvis wrote: post #5

Earlier quoted context omitted.

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

The "not a real language" critique often comes from those without any knowledge of Scheme; "real" languages are like C# or Java, and maybe Python, even though the lisp-like properties of Python are not so much appreciated. As C# and Java have gotten more functional aspects over the years, I've seen less and less of the "not a real language" criticism.

[deleted]

Re: An opinionated view of the Tidyverse “dialect” of the R language

#65

Post by mespe »

FranzFerdiNaN wrote: post #61
mespe wrote: post #52

I teach and consult on R and data science. I had the privilege of learning R from one of R's core developers. My students often ask why I don't use the Tidyverse. The answer is because I don't need to - I can do everything the Tidyverse does and so much more in base R. This article only briefly touches on what I think is the biggest issue with the Tidyverse. The Tidyverse is incredibly limiting. The "tidy" workflow h…

"Meanwhile, none of the truly advanced R users I encounter use the Tidyverse." This is most likely because truly advanced R users have been using R since far before the Tidyverse existed. A whole new generation of R users is being brought up with the Tidyverse, so im curious to see how the situation will be in 10 years time.

This is not entirely true - many are of the same "generation" as me. To be clear, I don't actually consider myself an advanced user. To me, many of these advanced users are pushing against the limits of the language in a way you don't see with run of the mill data science.

That said, I have heard my mentor say "I don't get the point of - I did this 20 years ago."

Re: An opinionated view of the Tidyverse “dialect” of the R language

#66

Post by tvladeck »

minimaxir wrote: post #55

From personal experience, I learned R during college as a part of my stats courses, before tidyverse even existed. After doing a few personal projects using base R with complex manipulations, I would have quit using R entirely if it were not for tidyverse/dplyr. I'm a pragmatic software engineer: I prefer to get a data analysis job as fast as necessary, and IMO there isn't much merit in making things more complicated…

> a lot of redundant df assignments

either that or a profusion of variables going into your global environment, which is a massive pain to track!

Re: An opinionated view of the Tidyverse “dialect” of the R language

#67

Post by jrumbut »

hardboiled wrote: post #62
jrumbut wrote: post #41

Earlier quoted context omitted.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly). I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr. That said it is hard…

Tidyverse code is not write-only; it is designed to be mostly read-only where the level of abstraction is at the level of the domain, which in this case is data frames and data pipelines akin to the same constructs in relational algebra/SQL or data processing (map/reduce). Correct usage requires learning the right abstractions, but fortunately these abstractions are shared across language communities and frameworks.…

I understand that's the goal, but it hasn't been what I've seen from user's code. I think the issue, which the article describes, is that the tidyverse has too large a surface area for something aimed at very smart people who are not software engineers and don't have time to become one.

So they end up with a subset of functionality they can get stuff done with, but then they need to collaborate with someone who uses a different subset. That can be messy.

I think in an environment where everyone learns R from the same course/book/MOOC/whatever (or have done lots of different sorts of programming) and the organization can impose a style guide the tidyverse approach would be great, but when you have people coming from all sorts of places and backgrounds I don't think it's a good fit.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#68

Post by thom »

Bootvis wrote: post #48
thom wrote: post #20

Earlier quoted context omitted.

Yeah, I'm surprised we're having performance arguments about these two libraries with mostly undefined performance characteristics which both run on a single-threaded runtime.

data.table does multithreading for a number of common operations. Running in parallel (not multithreaded) is quite well supported.

data.table’s OpenMP stuff is pretty haphazard, and can’t parallelise anything that calls back into R code. And anything outside of this involving forking lots has just been painful every time I’ve seen it, and again, way slower than doing it on a more performant platform up front.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#69

Post by thom »

jrumbut wrote: post #41
thom wrote: post #9

Wow, the idea that the Tidyverse is somehow burdened with an overabundance of computer science or functional programming is just hilarious to this programmer who has had to maintain, debug and optimise other people's R code.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly). I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr. That said it is hard…

This is more a symptom of the fact that nobody has told R programmers that they are software engineers and the thing that they’re creating is software first, and not just the plot that comes at the end. No library can really help that.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#70

Post by jrumbut »

thom wrote: post #69
jrumbut wrote: post #41

Earlier quoted context omitted.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly). I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr. That said it is hard…

This is more a symptom of the fact that nobody has told R programmers that they are software engineers and the thing that they’re creating is software first, and not just the plot that comes at the end. No library can really help that.

You're absolutely right with this diagnosis.
Post reply on HN