Live data from Hacker News

A newcomer’s (angry) guide to R

arrgh.tim-smith.us

121–130 of 232 posts

Re: A newcomer’s (angry) guide to R

#122
post #19

HN is predisposed to hate R because everyone here is coming from a "real" programming context. Their concerns are generally valid, but they should keep in mind a lot of people using do not have a software development background and do not care that the language is not elegantly designed: they just want to get analytical work done. In that respect, R is far, far superior to Python. Even something as simple as installi…

No, you are wrong. R is terrible , and especially so for non-professional programmers, and it is an absolute disaster for the applications where it routinely gets used, namely statistics for scientific applications. The reason is its strong tendency to fail silently (and, with RStudio, to frequently keep going even when it does fail.) As a result, people get garbage results without realizing , and if they're unlucky,…

You're right and wrong: R is a disaster when you want to write programs as you would in a real programming language. R is an excellent choice for what it is used most of the time by these people whose education/training isn't related to programming: interactive analysis of data and (maybe) writing prototypes.

Re: A newcomer’s (angry) guide to R

#123

Earlier quoted context omitted.

It's also worth noting that R becomes much more pleasurable with the Tidyverse libraries. The pipe alone makes everything more readable. I'm also coming from more of an office setting where everything is in Excel. I've used R to reorganize and tidy up Excel files a lot. Ggplot2 (part of the Tidyverse) is also fantastic for plotting, the grammar of graphics makes it really easy to make nice and slightly complex graphs…

Tidy features (like pipes) are detrimental to performance. The best things R has going for it are data.table, ggplot, stringr, RMarkdown, RStudio, and the massive, unmatched breadth and depth of special-purpose statistics libraries. Combined, this is a formidable and highly performant toolset for data analytics workflows, and I can say with some certainty that even though “base Python” might look prettier than “base…

Can you backup your performance claim? The last time I checked a benchmark that compared dplyr vs DT, DT didn't come out as a winner.

Re: A newcomer’s (angry) guide to R

#124
post #19

HN is predisposed to hate R because everyone here is coming from a "real" programming context. Their concerns are generally valid, but they should keep in mind a lot of people using do not have a software development background and do not care that the language is not elegantly designed: they just want to get analytical work done. In that respect, R is far, far superior to Python. Even something as simple as installi…

One common thing across most the "real" programming languages makes them unfit for data work: 0-based indexing. It is just ridiculous to call the first row in a data set as 0th row, and the last row as (n-1)th row. It does not make any sense for data analytic work.

I think this comment reflects the fact that a lot of pen and pencil linear algebra/stats/econometrics all uses one-based indexing.

There are a few times where I’ve had to formally write out a zero-based indexing scheme of a given expression because python indexing can seem so weird in this case. For example, “lag zero” just sounds like a funny way of talking about lag 1 (to me). Of course, if you’re predicting y_t+1 then it sort of makes sense that y_t[+0] would be paired with B_0.

Then there is the whole thing about how, say, range(5) will return 0,1,2,3,4 and NOT 5.

All of this makes sense once you use python for a while but if you spend most of your time writing with pencil then it will probably take some adjustment.

Re: A newcomer’s (angry) guide to R

#125
post #19

HN is predisposed to hate R because everyone here is coming from a "real" programming context. Their concerns are generally valid, but they should keep in mind a lot of people using do not have a software development background and do not care that the language is not elegantly designed: they just want to get analytical work done. In that respect, R is far, far superior to Python. Even something as simple as installi…

No, you are wrong. R is terrible , and especially so for non-professional programmers, and it is an absolute disaster for the applications where it routinely gets used, namely statistics for scientific applications. The reason is its strong tendency to fail silently (and, with RStudio, to frequently keep going even when it does fail.) As a result, people get garbage results without realizing , and if they're unlucky,…

While I kind of want to agree with you, I just don't see a better alternative. Do you really want biochemists to have to deal with the horrors of C compilation? In production code I'm very glad my makefile tells clang to fail on absolutely anything, but is that the best we can do? Other commenters have pointed out ways to avoid dangerous things like integer division, but if you think R is hostile then please offer a tenable alternative. The only ones I can think of are python and Matlab, and both are even worse for the intended use.

Yes, R is not my preferred language for anything heavy-duty, but I would guess ~95% of R usage is on datasets small enough to open in excel, and that is where the language truly shines (aside from being fairly friendly to non-programmers).

So yes, there are some problems with R, but what are your proposed improvements? Because if I have to analyze a .csv quickly, I'm going for R most of the time.

Re: A newcomer’s (angry) guide to R

#126
post #106

Earlier quoted context omitted.

It seems needlessly confusing to me to refer to the first number in a series as the 0th number. 0-based indexing is only good for offset counting, which is very much based on having a mental model based on pointer arithmetic for a number sequence.

Which years are in the decade the Twenties?

Yeah years start at 0. But that's because it measures the offset from the beginning of the calendar. You can similarly expand this to all distance based measurements. But that is completely different from counting, which shouldn't be conflated with distances. People always say the first of some sequence and only people who care about 0-based indexing tries to spread the 0th of some sequence meme.

Re: A newcomer’s (angry) guide to R

#127
post #97

Earlier quoted context omitted.

One common thing across most the "real" programming languages makes them unfit for data work: 0-based indexing. It is just ridiculous to call the first row in a data set as 0th row, and the last row as (n-1)th row. It does not make any sense for data analytic work.

I'm not sure I understand this, and I'm genuinely interested in why it would be. I find zero indexing logical: zero is the first natural number and is thus a fine candidate for being the first ordinal. In my experience most mathematical series lose nothing in terms of elegance or readability by being indexed from zero instead of using more traditional indexing from one.

Generally I've found you carry around fewer n +/- 1 type expressions when you index from 1. Also, most applied math papers I've read index from 1 and that makes implementing them a lot easier.

Re: A newcomer’s (angry) guide to R

#128

Earlier quoted context omitted.

No, you are wrong. R is terrible , and especially so for non-professional programmers, and it is an absolute disaster for the applications where it routinely gets used, namely statistics for scientific applications. The reason is its strong tendency to fail silently (and, with RStudio, to frequently keep going even when it does fail.) As a result, people get garbage results without realizing , and if they're unlucky,…

Silent failure and continuing to run on errors are common in interpreted languages. SAS has similar issues, most RDBMSs will continue to process queries after failures. It’s something you need to explicitly guard against.

Are you sure about "most RDBMSs"? With the exception of SQLite and older versions of MySQL, all the databases that I've used are strict and fail the query immediately on error, will generally prevent silently dropping or truncating data, etc.

I'm one of the original authors of Presto, a distributed SQL engine for analytics on big data. From the beginning, we've been careful to follow the SQL standard and do everything possible to either return the correct answer or fail the query. For example, an addition or sum aggregation on an integer will fail on overflow rather than silently wrapping.

Returning an incorrect answer or silently corrupting your data is the worst thing a database can do.

Re: A newcomer’s (angry) guide to R

#129

Earlier quoted context omitted.

No, you are wrong. R is terrible , and especially so for non-professional programmers, and it is an absolute disaster for the applications where it routinely gets used, namely statistics for scientific applications. The reason is its strong tendency to fail silently (and, with RStudio, to frequently keep going even when it does fail.) As a result, people get garbage results without realizing , and if they're unlucky,…

While I kind of want to agree with you, I just don't see a better alternative. Do you really want biochemists to have to deal with the horrors of C compilation? In production code I'm very glad my makefile tells clang to fail on absolutely anything, but is that the best we can do? Other commenters have pointed out ways to avoid dangerous things like integer division, but if you think R is hostile then please offer a…

> I just don't see a better alternative

Julia?

Re: A newcomer’s (angry) guide to R

#130
post #106

Earlier quoted context omitted.

It seems needlessly confusing to me to refer to the first number in a series as the 0th number. 0-based indexing is only good for offset counting, which is very much based on having a mental model based on pointer arithmetic for a number sequence.

Which years are in the decade the Twenties?

Which days are in a Month? You dont start with june 0th and end with june 29th?
Post reply on HN