Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An opinionated view of the Tidyverse “dialect” of the R language

github.com

Post reply on HN
Print view View original

101–110 of 156 posts

Re: An opinionated view of the Tidyverse “dialect” of the R language

#101

Post by hadley »

Bootvis wrote: post #49
jhbadger wrote: post #11

At least in my case, although I do like some of the tidyverse (ggplot is tidyverse after all, and probably my favorite plotting system in any language), I find the "tibble" data structure and its disdain for row names really annoying. Many, many, packages which I use expect a data frame with row names and likewise output such. So I am constantly converting back and forth between tibbles and data frames.

The claim that ggplot2 is tidyverse can be disputed. It existed before and works fine outside of it.

You are of course free to dispute what is in the tidyverse, but I pretty strongly believe that it is part of the tidyverse

Re: An opinionated view of the Tidyverse “dialect” of the R language

#102

Post by nonbel »

roenxi wrote: post #44

He is glossing over the major issue with base R, which is its tendancy to switch data types in a way that appears random to new users. As an undergrad I spent nights literally on the verge of tears debugging R code where the types had been mucked up by R's bizarre semantics. Claiming that data[foo] is the same as filter(data, foo) is not correct - the [] and [[]] operators have a lot of strange side effects depending…

> "Dr. Matloff is grossly underestimating how painful it is for beginners to, eg, figure out when they are dealing with a column of strings vs a column of factors."

This is just sapply(df, class)... Is this not found in R 101 material that beginners are likely to find?

Re: An opinionated view of the Tidyverse “dialect” of the R language

#103

Post by bsg75 »

phonebucket wrote: post #85
minimaxir wrote: post #55

From personal experience, I learned R during college as a part of my stats courses, before tidyverse even existed. After doing a few personal projects using base R with complex manipulations, I would have quit using R entirely if it were not for tidyverse/dplyr. I'm a pragmatic software engineer: I prefer to get a data analysis job as fast as necessary, and IMO there isn't much merit in making things more complicated…

With regards to the usefulness of piping with dplyr: the post does express its admiration for data.table as the alternative for dplyr. In data.table, chaining operations together is standard and as simple as a set of brackets to contain the next operation. Both are preferable to pandas though (joke not troll).

There are nuances that to me make R data frames, especially with Magrittr, simpler to work with over Pandas. Pandas indexing always seems to get in the way.

I have been using Python a lot longer than R, but the Pandas syntax is taking longer to internalize.

Maybe it's just me?

Re: An opinionated view of the Tidyverse “dialect” of the R language

#104

Post by wodenokoto »

bsg75 wrote: post #103
phonebucket wrote: post #85

Earlier quoted context omitted.

With regards to the usefulness of piping with dplyr: the post does express its admiration for data.table as the alternative for dplyr. In data.table, chaining operations together is standard and as simple as a set of brackets to contain the next operation. Both are preferable to pandas though (joke not troll).

There are nuances that to me make R data frames, especially with Magrittr, simpler to work with over Pandas. Pandas indexing always seems to get in the way. I have been using Python a lot longer than R, but the Pandas syntax is taking longer to internalize. Maybe it's just me?

No. Pandas is a powerful tool, but a poor api / user interface.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#105

Post by jointpdf »

stilley2 wrote: post #74

I recently tried to do some stuff in R with tidyverse, and was not a fan. I'm no expert, but the tidyverse's frequent use of nonstandard evaluation drove me crazy. It makes it so much more difficult to write functions encapsulating tidy functions. However, I never see people complain about this, so maybe I'm doing something wrong or not grokking something...

There is a new operator, {{ or “curly curly”, aimed at dealing with this. It’s part of the rlang (0.4) package and looks snazzy / easy to grasp.

Check it out: https://www.tidyverse.org/articles/2019/06/rlang-0-4-0/

Re: An opinionated view of the Tidyverse “dialect” of the R language

#106

Post by notafraudster »

cwyers wrote: post #98
notafraudster wrote: post #83

Earlier quoted context omitted.

I run a large scale national survey. We download the data from our survey platform. Survey respondents are asked 100+ questions. The questions change week to week and so the column names are not consistent. We exclude respondents who appear to be cheating the system (rushing through questions, straight-lining, skipping almost every question, etc.) As part of our completion check, we want to do a row-wise map of the d…

map() in purrr is functionally equivalent to lapply(). If you can do something in lapply, you can do it with map.

Right, which is why my example used its cousin, apply, instead of lapply. apply over the row margins of a data frame does not have an equivalent in tidyverse.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#107

Post by RosanaAnaDana »

hadley wrote: post #101
Bootvis wrote: post #49

Earlier quoted context omitted.

The claim that ggplot2 is tidyverse can be disputed. It existed before and works fine outside of it.

You are of course free to dispute what is in the tidyverse, but I pretty strongly believe that it is part of the tidyverse

Seeing as the tidyverse is pretty much your invention, its well within your purvue to define what is in and what is out. I do think, respectfully, that both the op and the general sentiment article have the right of it.

Data.table and data.table-esque notation represent such an improvement of tibble/dplyr, that within my company, we're making a concerted effort to purge all tidyverse packages from general use (less ggplot). When new developers come on, if they are coming from tidyverse, their first task will be something involving pipes and data.table. Tidyverse was fine in school. It doesn't pass muster in production, at least not in our work.

Data.table syntax is simpler, easier to read, easier to teach, and orders of magnitude faster. It plays nicer with other packages than the tidyverse (if it fits into a DF, it almost always fits into a DT, and i've never met a tibble that I didn't wish was a data.table), and since almost all of our datasets are 10's to 1000's of millions of lines long, the decision was really made for us.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#108

Post by RosanaAnaDana »

barcadad wrote: post #89
inputcoffee wrote: post #58

I was waiting for the critique... but I never quite saw it. Imho, the data problem Tidyverse is trying to solve is basically the ones we face in a database. So, select, join, inner join and so forth. Show me all the rows in this datatable where the 4th columm is larger than the 6th column and the number itself is odd. Something like that. There might be other ways to do it, but you want your select, filter, summarize…

His critique is more about the impact of the full ecosystem effect of the Tidyverse, not what you are referring to, which is just the dplyr semantics. The Tidyverse demands that it's many related packages use tidy data principles and lock users into that approach, which differs from base-R. Much of this discussion is really just a debate about dplyr and magrittr rather than the fragmentation that the broader tidyvers…

I think any one making the point of speed of development have seriously missed the boat. Lets be real: tibbles suck. Once you get the hang of data.table syntax for matrix operations, its superiority becomes impeccably clear.

I run a data science group a large geospatial company and we develop day in and day out in R and python. We've purged tidyverse as much as possible from all of our code base. We've moved completely over to data.tables, which make the vast majority of the tidyverse irrelevant.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#109

Post by RosanaAnaDana »

jointpdf wrote: post #105
stilley2 wrote: post #74

I recently tried to do some stuff in R with tidyverse, and was not a fan. I'm no expert, but the tidyverse's frequent use of nonstandard evaluation drove me crazy. It makes it so much more difficult to write functions encapsulating tidy functions. However, I never see people complain about this, so maybe I'm doing something wrong or not grokking something...

There is a new operator, {{ or “curly curly”, aimed at dealing with this. It’s part of the rlang (0.4) package and looks snazzy / easy to grasp. Check it out: https://www.tidyverse.org/articles/2019/06/rlang-0-4-0/

Yuck.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#110

Post by sarah180 »

nerdponx wrote: post #25
Bootvis wrote: post #5

Earlier quoted context omitted.

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

The R standard library is kind of weird and hard to use for "general purpose" programming. It's very clearly a domain specific language. Also, it's unbelievably slow for basic operations like looping, function calls, and variable assignment. It's literally orders of magnitude slower than Python (I've tested it). Unlike its fellow C-flavored-Lisp Javascript, R retains an extreme level of homoiconicity, Which apparentl…

it's ultimately and fundamentally a domain-specific scripting language

A domain-specific rather than general-purpose programming language? Most definitely. A scripting language? I would dispute that. Scripting languages are for automating the execution of sequences of tasks that you'd otherwise run independently.

Post reply on HN