Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A newcomer’s (angry) guide to R

arrgh.tim-smith.us

Post reply on HN
Print view View original

191–200 of 232 posts

Re: A newcomer’s (angry) guide to R

#191

Post by riskneutral »

stewbrew wrote: post #123
riskneutral wrote: post #45

Earlier quoted context omitted.

Tidy features (like pipes) are detrimental to performance. The best things R has going for it are data.table, ggplot, stringr, RMarkdown, RStudio, and the massive, unmatched breadth and depth of special-purpose statistics libraries. Combined, this is a formidable and highly performant toolset for data analytics workflows, and I can say with some certainty that even though “base Python” might look prettier than “base…

Can you backup your performance claim? The last time I checked a benchmark that compared dplyr vs DT, DT didn't come out as a winner.

Here is one example: https://appsilondatascience.com/blog/rstats/2017/03/02/r-fas...

I also base this on my own experience. I typically work with 2-3 million row datasets. I found that doing certain data operations was quite slow in plyr but a lot faster in data.table. It’s possible that if I had spent time reordering my plyr pipelines and filtering out unneeded columns or rows, then it would have worked better. However, data.table doesn’t require such planning ahead and thinking about what columns/rows you need to send to the next operation in a pipeline, because multiple operations can be executed from a single data.table call, and the underlying C library is able to make optimized decisions (like dropping columns not requested in the query), similar to an in-memory SQL database. So between dealing with slow code while doing interactive analysis, and/or having to spend time hand-optimizing dplyr pipelines, I found data.table to be a significant improvement in productivity (other than the one-time effort of having to rewrite a few internal packages/scripts to use data.table instead of dplyr)

Re: A newcomer’s (angry) guide to R

#192

Post by lispm »

vertere wrote: post #190
int_19h wrote: post #167

Earlier quoted context omitted.

> R and Lisp are hardly alike even if it was inspire by it. It's like saying Erlang and Prolog is very similar. If you want learn FP do it in Erlang, Lisp, Haskell, etc.. Don't do it in R, it's half baked. They are very alike in the underlying core design, not in how you use them. In R, everything is an expression, and every expression is a function call. Even things like assignments, if/else, or function definitions…

R certainly has a lispish code-as-data element to it, but it seems like it has some serious flaws. Don't most lisps have functions and macros as separate constructs? R has functions, but with some mucking around you can make them do macro-type stuff. Then people write these half-function, half-macro things (e.g. "non-standard evalation") that tend to break composability, either totally or sometimes only in edge cases…

Something like that would be called a FEXPR in Lisp.

https://en.wikipedia.org/wiki/Fexpr

Re: A newcomer’s (angry) guide to R

#193

Post by riskneutral »

extr wrote: post #88
vijucat wrote: post #68

Earlier quoted context omitted.

Answering questions in a rapid, interactive way (, while using C to be efficient enough that one can run it on millions of rows): # Given a dataset that looks like this… > head(dt, 3) mpg cyl disp hp drat wt qsec vs am gear carb name 1: 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4 Mazda RX4 2: 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4 Mazda RX4 Wag 3: 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1 Datsun 710 # What's the mean hp an…

Completely agree. dplyr is nice enough but the verbose style gets old fast when you're trying to use it in an interactive fashion. imo data.table is the fastest way to explore data across any language, period.

I strongly agree, having worked quite a bit in several languages including Python/NumPy/Pandas, MATLAB, C, C++, C#, even Perl ... I am not sure about Julia, but last time I looked at it, the language designers seemed to be coming from a MATLAB type domain (number crunching) as opposed to an R type domain (data crunching), and so Julia seemed to have a solid matrix/vector type system and syntax, but was missing a data.table style type system / syntax.

Re: A newcomer’s (angry) guide to R

#194

Post by jandrese »

txcwpalpha wrote: post #147
jandrese wrote: post #66

Earlier quoted context omitted.

Some of it is undoubtedly hyperbole, but: > Index vectors like a[1] … a[4]. All indexing in R is base-one. Note that no error is thrown if you try to access a[0]; it always returns an atomic vector of the same type but of length zero, written like numeric(0) That's serious WTF right there. In general a lot of the complaints revolve around the language making error handling unnecessarily difficult which is something t…

It's not WTF at all once you actually understand why it works that way and the benefits it provides. Subsetting in R allows for any index number to be retrieved, and if there is no value, it returns an empty value (that's what "numeric(0)" is: an empty numeric value). It's the same if you tried to access a[0] or a[90000] (in an array that doesn't have 90000 elements). This makes it easier to select multiple elements…

It has a different behavior when you access past the end of the array. That makes error checking more difficult since you have to test for both error conditions. 1 based indexing is a mistake, but compounding that by changing the error conditions is the WTF.

Having 1 based indexing is a design flaw that R shares with SQL. SQL's error handling also leaves much to be desired.

Re: A newcomer’s (angry) guide to R

#195

Post by Sean1708 »

int_19h wrote: post #167
digitalzombie wrote: post #115

I use R a lot and I have to say some of these comments are weird. 1. R and Lisp are hardly alike even if it was inspire by it. It's like saying Erlang and Prolog is very similar. If you want learn FP do it in Erlang, Lisp, Haskell, etc.. Don't do it in R, it's half baked. 2. R syntax is ugly with warts. But built in datatype like dataframe, factor type, NA (missing value notion) value, make this language much better…

> R and Lisp are hardly alike even if it was inspire by it. It's like saying Erlang and Prolog is very similar. If you want learn FP do it in Erlang, Lisp, Haskell, etc.. Don't do it in R, it's half baked. They are very alike in the underlying core design, not in how you use them. In R, everything is an expression, and every expression is a function call. Even things like assignments, if/else, or function definitions…

Do you have an example of what R would look like without the C-like syntactic sugar? It doesn't need to be complex, I'm just intrigued about what it might look like.

Re: A newcomer’s (angry) guide to R

#196

Post by newen »

pfortuny wrote: post #170
ScottBurson wrote: post #129

Earlier quoted context omitted.

> I just don't see a better alternative Julia?

uf, I’m a working mathematician and just reading julia’s documentarion makes me dizzy... I do not think it fits this use case.

This might be a better intro http://ucidatascienceinitiative.github.io/IntroToJulia

Re: A newcomer’s (angry) guide to R

#197

Post by rpier001 »

roenxi wrote: post #146
rpier001 wrote: post #99

Earlier quoted context omitted.

I think what you're saying is mostly on point. I wanted to share a couple possible balms for your bugbears. For attach metadata to an anything, why not use attributes()/attr() or the tidy equivs? Isn't that what it is for? It might not make you feel much better, but data.frame is just a special list, c.f. is.list(data.frame()). So, if you don't want to use the connivence layers for data.frame you can just pretend it…

I wrote a function once for a friend that modified the enclosing environment, and changed + so that sometimes it added two numbers together and sometimes it added to numbers and an extra 1 just to be helpful. I can sort myself out, but thanks for the thoughts. The issue is that I learn these things /after/ R does something absolutely off the wall with its type system. And a lot of my exposure comes from using other p…

scale() for example uses attributes to hold on to the parameters used for scaling. Most packages that use attributes provide accessor functions so that the useR doesn't need to concern themselves with how the metadata are stored. I'll grant that people do tend to use lists because the access semantics are easier.

Re: A newcomer’s (angry) guide to R

#198

Post by angry_octet »

int_19h wrote: post #166
rspeer wrote: post #149

Earlier quoted context omitted.

Ubuntu doesn't ship with pip or virtualenv. In fact it ships with a version of Python where the built-in equivalent to virtualenv, pyvenv, is explicitly disabled. So you have to install extra Python packages, as root. You have to have that Python experience that guides you to install as few of them as you can, just enough so you can get started with a virtualenv, so you don't end up relying on your system Python envi…

That sounds like a major problem with Ubuntu, rather than with Python or pip. On Windows, meanwhile, the standard Python installer gets all this set up properly in like three clicks. Better yet, because it installs per-user by default, "pip install" just works. And if you still choose to install it globally, it will fail, but it will tell you exactly what you need to do to make it work: Could not install packages due…

Don't despair, in the Anaconda installed with visual studio (now a default) you can't update or install packages without being admin! And if you install Anaconda again it merges the start menu entries and you can't tell which is which...

Re: A newcomer’s (angry) guide to R

#199

Post by pfortuny »

newen wrote: post #196
pfortuny wrote: post #170

Earlier quoted context omitted.

uf, I’m a working mathematician and just reading julia’s documentarion makes me dizzy... I do not think it fits this use case.

This might be a better intro http://ucidatascienceinitiative.github.io/IntroToJulia

Sure, thanks. I am always put off Julia because of the documentation.

Re: A newcomer’s (angry) guide to R

#200

Post by angry_octet »

acomjean wrote: post #25
extr wrote: post #19

HN is predisposed to hate R because everyone here is coming from a "real" programming context. Their concerns are generally valid, but they should keep in mind a lot of people using do not have a software development background and do not care that the language is not elegantly designed: they just want to get analytical work done. In that respect, R is far, far superior to Python. Even something as simple as installi…

As a datapoint in agreement. Some of the Biologists I work with love R. I had one tell me that its like how they think, and they weren't a python fan. I think R-Studio (an R based IDE that turns it kinda into a more excel like experience) where you can inspect the data in memory (including matrix data) and graph making is where it really helps bring people into the R language. And with a set of instructions anyone ca…

Anaconda and Jupyter is a much more friendly environment than straight writing .py files, much like r studio. It lacks the integrated debugging features, though vscode does provide some rudimentary assistance. I'd say it's superior overall, especially in regard to getting help and documentation.
Post reply on HN