Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A newcomer’s (angry) guide to R

arrgh.tim-smith.us

Post reply on HN
Print view View original

201–210 of 232 posts

Re: A newcomer’s (angry) guide to R

#201

Post by ChrisRackauckas »

riskneutral wrote: post #193
extr wrote: post #88

Earlier quoted context omitted.

Completely agree. dplyr is nice enough but the verbose style gets old fast when you're trying to use it in an interactive fashion. imo data.table is the fastest way to explore data across any language, period.

I strongly agree, having worked quite a bit in several languages including Python/NumPy/Pandas, MATLAB, C, C++, C#, even Perl ... I am not sure about Julia, but last time I looked at it, the language designers seemed to be coming from a MATLAB type domain (number crunching) as opposed to an R type domain (data crunching), and so Julia seemed to have a solid matrix/vector type system and syntax, but was missing a data…

Julia v0.7-alpha dropped and it has a new system for missing data handling. JuliaDB and DataFrames are two tabular data stores (the first of which is parallel and allows out-of-core for big data). This has changed pretty dramatically over the last year.

Re: A newcomer’s (angry) guide to R

#202

Post by ChrisRackauckas »

jghn wrote: post #132
ScottBurson wrote: post #129

Earlier quoted context omitted.

> I just don't see a better alternative Julia?

The biggest Julia cheerleaders gave up years ago. That doesn't mean the language is dead but it's not a great sign for a niche lang

The biggest Julia cheerleaders are using Julia.

Re: A newcomer’s (angry) guide to R

#203

Post by stewbrew »

riskneutral wrote: post #191
stewbrew wrote: post #123

Earlier quoted context omitted.

Can you backup your performance claim? The last time I checked a benchmark that compared dplyr vs DT, DT didn't come out as a winner.

Here is one example: https://appsilondatascience.com/blog/rstats/2017/03/02/r-fas... I also base this on my own experience. I typically work with 2-3 million row datasets. I found that doing certain data operations was quite slow in plyr but a lot faster in data.table. It’s possible that if I had spent time reordering my plyr pipelines and filtering out unneeded columns or rows, then it would have worked better. Howe…

Thanks for the reference. Why don't you keep your data in a DB? I load almost anything that isn't a small atomic data frame into a RDBMS.

BTW one thing that always made me avoid DT (I even preferred sqldf before dplyr was created) was its IMHO weird syntax. I always found the syntax of (d)plyr much more convenient. ATM it seems to me that dplyr has won the contest of alternative data management libraries. I cannot remember when I last read a blog post, article, or book that preferred DT over dplyr. I'm old enough to have learned that wrt libraries, it's wise to follow the crowd.

Re: A newcomer’s (angry) guide to R

#204

Post by dnautics »

pfortuny wrote: post #170
ScottBurson wrote: post #129

Earlier quoted context omitted.

> I just don't see a better alternative Julia?

uf, I’m a working mathematician and just reading julia’s documentarion makes me dizzy... I do not think it fits this use case.

That's ironic because I find Julia's documentation to be the second most clear documentation I've seen (after elixir). Notation wise, Julia is the most comfortably close to mathematics (APL is closer, but it's a write only language). I'm not a working mathematician, though i did graduate with a rather theory based math degree.

Re: A newcomer’s (angry) guide to R

#205

Post by psychometry »

maxander wrote: post #91
extr wrote: post #19

HN is predisposed to hate R because everyone here is coming from a "real" programming context. Their concerns are generally valid, but they should keep in mind a lot of people using do not have a software development background and do not care that the language is not elegantly designed: they just want to get analytical work done. In that respect, R is far, far superior to Python. Even something as simple as installi…

No, you are wrong. R is terrible , and especially so for non-professional programmers, and it is an absolute disaster for the applications where it routinely gets used, namely statistics for scientific applications. The reason is its strong tendency to fail silently (and, with RStudio, to frequently keep going even when it does fail.) As a result, people get garbage results without realizing , and if they're unlucky,…

You haven't been working with scientists very long, have you? I'm guessing you're also only a very recent CS grad. You're criticizing a language based on the behavior of certain people who use the language, rather than criticizing the language itself.

For many years before you mounted your high horse, scientists were writing equally shitty code in Perl. When they've moved on from R, they'll write shitty code in some other language.

Re: A newcomer’s (angry) guide to R

#206

Post by stewbrew »

riskneutral wrote: post #191
stewbrew wrote: post #123

Earlier quoted context omitted.

Can you backup your performance claim? The last time I checked a benchmark that compared dplyr vs DT, DT didn't come out as a winner.

Here is one example: https://appsilondatascience.com/blog/rstats/2017/03/02/r-fas... I also base this on my own experience. I typically work with 2-3 million row datasets. I found that doing certain data operations was quite slow in plyr but a lot faster in data.table. It’s possible that if I had spent time reordering my plyr pipelines and filtering out unneeded columns or rows, then it would have worked better. Howe…

About that article: I assume that DT uses an index for that column while dplyr does a full search. If that's really the case the result wouldn't be that much a surprise.

Re: A newcomer’s (angry) guide to R

#207

Post by dnautics »

bograt wrote: post #97
educationdata wrote: post #94

Earlier quoted context omitted.

One common thing across most the "real" programming languages makes them unfit for data work: 0-based indexing. It is just ridiculous to call the first row in a data set as 0th row, and the last row as (n-1)th row. It does not make any sense for data analytic work.

I'm not sure I understand this, and I'm genuinely interested in why it would be. I find zero indexing logical: zero is the first natural number and is thus a fine candidate for being the first ordinal. In my experience most mathematical series lose nothing in terms of elegance or readability by being indexed from zero instead of using more traditional indexing from one.

Generally 1 is considered the first natural number, except bourbaki.

The reason for this is the set {1...n} has order n, but the set {0...n} has order n+1, so you get lots of off by n+1 inelegancies or errors when order is important. It's better to be explicit at the set level when you need an {0...n-1} set, because usually the order gets passed around to later expressions and not the set element, so there's less algebra.

Zero indexing is great when your index is an offset, as it is for true arrays.

Re: A newcomer’s (angry) guide to R

#208

Post by dnautics »

sametmax wrote: post #182
carlmr wrote: post #155

Earlier quoted context omitted.

>Just to be fair, C is also a horrible name. On the other end of the spectrum - Julia and Rust are excellent names for a programming language because they're unique in the context of programming. Funny enough I most often get wrong results when googling something Rust related, because there's a town called Rust (Germany), so Google pushes the location based results up, and rust is also, well, oxidized metal, so somet…

Which is why we use "rustlang" and "golang" for keywords, tags, and search terms everywhere. Maybe they should start using Rlang. Although it's going to be annoyingly close to erlang

Funny story, I once worked extensively with early Julia in a scientific setting where the hr directors name was Julia Lang, and we joked that if someone checked packets they would think I was stalking her.

Re: A newcomer’s (angry) guide to R

#209

Post by ChrisRackauckas »

dnautics wrote: post #204
pfortuny wrote: post #170

Earlier quoted context omitted.

uf, I’m a working mathematician and just reading julia’s documentarion makes me dizzy... I do not think it fits this use case.

That's ironic because I find Julia's documentation to be the second most clear documentation I've seen (after elixir). Notation wise, Julia is the most comfortably close to mathematics (APL is closer, but it's a write only language). I'm not a working mathematician, though i did graduate with a rather theory based math degree.

The documentation is fine but IMO written more for developers. We do need more mathematical-based introductions which introduce the right packages for working mathematicians. I am a working mathematician myself and find Julia to be the perfect language for it because its abstraction is on actions instead of on data representations which fits things like functional analysis extremely well.

Things that are OO based like C++ and Python are pretty bad at representing math because they put forward an idea of the actual representation (the object) as what matters, instead of the actions it performs (the function overloads). This may be good for some disciplines, but in a mathematical algorithm I really don't care what kind of matrix you gave me for `A`, I just want you to do the efficient `Ax=b` solve and have the action of the solver choose the appropriate method to abstract away the data. In Python you'd have to tell it to use the SciPy banded matrix solver, in Julia your generic ODE solver will automatically use the banded matrix solver when it's a banded matrix. This then allows for a composibility where the user overloads the primitive operations on their type, and your generic algorithm works on any data representation. This matches the workflow of math where an algorithm is proven on L2 functions, not on functions represented with column-wise indexing and ...

Re: A newcomer’s (angry) guide to R

#210

Post by d_watt »

fermienrico wrote: post #95
thousandautumns wrote: post #63

This is a stupendous example of someone going overboard on their criticisms in order to grandstand. R may not be the most "beautiful" language in a general perspective, but it certainly is more beautiful than Python when it comes to actual data analysis. There is nothing in R that is as ugly as even the best implemented pandas, numpy, and matplotlib code. All of the options in Python, which is generally pointed to as…

R is a poor name, whether you can google it or not. The name can get lost in the minefield of text on the internet. Just because you never had any issues with google R does not make it any better. I have had many issues googling R and it always makes me second guess if this thread is about R language at all. On SO, I have to check if R is tagged. R is a terrible name and it is not up for a debate. Whenever you name a…

To be fair, when both of these came out, SEO wasn't a thing.
Post reply on HN