Live data from Hacker News

An opinionated view of the Tidyverse “dialect” of the R language

github.com

41–50 of 156 posts

Re: An opinionated view of the Tidyverse “dialect” of the R language

#41
post #9

Wow, the idea that the Tidyverse is somehow burdened with an overabundance of computer science or functional programming is just hilarious to this programmer who has had to maintain, debug and optimise other people's R code.

I think this is what the author is getting at, that correct tidyverse usage requires greater experience and knowledge and creates difficult to debug and optimize code when used incorrectly (and is easy to use incorrectly).

I tend to agree. I've found tidyverse code has a write-only quality to it. Since I'm going to see more of it I plan to dive into the inner workings of at least dyplr and purr.

That said it is hard to deny what Hadley Wickham has done for the R community and I can't write off the idea that he sees a bigger picture I'm missing.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#42
post #7

Earlier quoted context omitted.

I agree; which is why I generally get confused when I see that criticism. This is only an assumption though; maybe its just a vocal minority that don't like R or I only notice the negative comments. Who knows

For what it’s worth, I am a JavaScript programmer and I feel like HN shits on JS All. The. Time. Some people just like to shit on languages. I think it’s just insecurity. They worry a lot about whether they’re a good enough developer, so they try hard to be on the “most advanced” platform they can figure out, and then need to talk down to the other platforms. Secure, talented, older developers, in my experience, tend…

Javascript is just a terrible language and programming environment though. Anyone can criticize people who disagree with them by calling them "insecure". It's a meaningless argument.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#43
post #6

I think the same thing is happening in Python for data science due to Pandas. It’s a great package, but many data scientists are only able to manipulate data using Pandas, and know little “base” Python.

That sounds incredible to me as using pandas without falling into the traps of chained indexing seems so much harder than 90% of Python.

I've been planning to go from "can work it out with copious examples" to "knows when to use apply, transform, etc. off the cuff" level of knowledge in Pandas - do you have any suggestions on good resources on this?

I'd like to understand better why the data structures work the way they do and thus have an intuition on what operations to use when. The O'Reilly Python Data Science Handbook[0] seems like it might be useful here, but I'm not sure if it is still up to date.

[0] https://www.oreilly.com/library/view/python-data-science/978...

Re: An opinionated view of the Tidyverse “dialect” of the R language

#44
He is glossing over the major issue with base R, which is its tendancy to switch data types in a way that appears random to new users. As an undergrad I spent nights literally on the verge of tears debugging R code where the types had been mucked up by R's bizarre semantics. Claiming that data[foo] is the same as filter(data, foo) is not correct - the [] and [[]] operators have a lot of strange side effects depending on what argument is passed in. It is like saying map() and a for loop are the same because everything a map can do a for loop can do. That is true, but the single-mindedness of map is a major selling point because you know it is only going to do certain specific things. The issues here will disproportionately affect beginners who are likely to accidentally pass in the wrong thing - it needs to fail with an error or in a predictable way to best help them. Hadley's code fails sensibly.

One of the major innovations of the tibble package is that it puts the types of each column into the headings when it prints the table out. Dr. Matloff is grossly underestimating how painful it is for beginners to, eg, figure out when they are dealing with a column of strings vs a column of factors. If a beginner copies code to someone else for help, the other person can actually tell what types are involved! It is like magic compared to base R.

Given how ad-hoc the R community is and the paucity of people with programmer backgrounds, these minor usability improvements on controlling types are important. And anybody who cares about efficiency is using the wrong language.

EDIT Also, this comment that RStudio is bypassing an Open Source project's leadership - that is one fo the major features of open source. It is usually a healthy sign that the project leadership are lost in the weeds. I recall a situation with broad parallels in the history of GCC for example; it didn't kill the project although I believe a bad codebase died.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#45
post #6

I think the same thing is happening in Python for data science due to Pandas. It’s a great package, but many data scientists are only able to manipulate data using Pandas, and know little “base” Python.

You can’t really do any data analysis without pandas in python though.

In fact python is a terrible language and the only reason anyone should use it is for access to sklearn, scikit, pytorth, and pandas.

Hopefully Julia will be able to unseat python for data analysis in the future.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#46
at the end of the comments of the post http://varianceexplained.org/r/teach-tidyverse/

compare base R:

  var1 
Now hadley solution comment: You can now express that sort of thing fairly elegantly with dplyr:

  var1 
Thats a lot of added complexity to be able to use column names, and to top it all, the use of non standard evaluation make things very complex when one go a little away from basic EDA.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#47
post #5

Earlier quoted context omitted.

I don’t even understand the not a real language critique, R: - is based on scheme, a HN favourite; - integrates very well with C++ through RCpp allowing you to do whatever you want. 9/10 times someone already went through the trouble for you.

The R standard library is kind of weird and hard to use for "general purpose" programming. It's very clearly a domain specific language. Also, it's unbelievably slow for basic operations like looping, function calls, and variable assignment. It's literally orders of magnitude slower than Python (I've tested it). Unlike its fellow C-flavored-Lisp Javascript, R retains an extreme level of homoiconicity, Which apparentl…

The standard lib is limited for general programming, but many standard libraries are and often don’t have a package repository as good as CRAN.

It seems the Graal people managed to make R quite fast and is still being developed. Is there another faster? Any way, check out my blog post:

http://bootvis.nl/fastr-sudokus/

However, fastR is not a drop in replacement yet.

In any case, for me professionally, R slowness is never a problem. data.table does the heavy lifting.

What is a confusing mess, is the object system.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#48
post #20
post #16

Earlier quoted context omitted.

Depending on the size of your data, you might not care that dplyr is slower than data.table. If you're better at writing/composing dplyr, you can often make up the speed difference between the two in terms of the savings in time spent writing and reading code. And if your data is that large, there are solutions like dbplyr out there to run dplyr code on various backends and offload the computation outside of R.

Yeah, I'm surprised we're having performance arguments about these two libraries with mostly undefined performance characteristics which both run on a single-threaded runtime.

data.table does multithreading for a number of common operations. Running in parallel (not multithreaded) is quite well supported.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#49

At least in my case, although I do like some of the tidyverse (ggplot is tidyverse after all, and probably my favorite plotting system in any language), I find the "tibble" data structure and its disdain for row names really annoying. Many, many, packages which I use expect a data frame with row names and likewise output such. So I am constantly converting back and forth between tibbles and data frames.

The claim that ggplot2 is tidyverse can be disputed. It existed before and works fine outside of it.

Re: An opinionated view of the Tidyverse “dialect” of the R language

#50

at the end of the comments of the post http://varianceexplained.org/r/teach-tidyverse/ compare base R: var1 Now hadley solution comment: You can now express that sort of thing fairly elegantly with dplyr: var1 Thats a lot of added complexity to be able to use column names, and to top it all, the use of non standard evaluation make things very complex when one go a little away from basic EDA.

Well, strictly literately to this case the tidyverse command would be:

mtcars %% mutate(pounds = wt / 1000)

Which is obviously simpler, and quite likely what a beginner is actually trying to do.

The real complaint - "tidyverse doesn't let us use variables to name columns!" - is fair enough, following quasiquotation is hard work. If you need that, drop back to base R. However, it is harder to teach because all the common operations (select, filter, etc) will involve some combination of []/[[]] and relatively hard to figure out which one a piece of code is trying to do.

Except for the lucky few who have brains wired to think in terms of arrays and database relations, someone learning the language is unlikely to thank you for that.

Post reply on HN