What's Next for R? • Hacker News Board

Re: What's Next for R?

#51

Post by phillc73 » Sat, Dec 28, 2019, 10:01 AM UTC

xiaodai wrote: post #12

meztez wrote: post #11

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible. I get that R is not for everyone but used correctly it is a beast. Now thi…

For larger-than-RAM data I would recommend diskframe.com It uses dplyr and data.table syntax to manipulate data on disk

I've not used diskframe.com, but from experience can recommend the 'fst'[1] file format with 'fsttable'[2] for reading on disk data tables.

[1] https://github.com/fstpackage/fst

[2] https://github.com/fstpackage/fsttable

Re: What's Next for R?

#52

Post by jhanschoo » Sat, Dec 28, 2019, 10:21 AM UTC

CreRecombinase wrote: post #33

thegginthesky wrote: post #30

When I used R in University (majored in Applied Mathematics and Statistics) I was always awestruck at how every sort of novel modeling technique from GLM, to Beta Regressions, to GARCH, is all easily accessible for free, with proper academic paper and documentation, and with a cohesive standard support. It was really useful to be able to apply most theory I was learning to actual research datasets. This is what I mis…

What didn't you like about the packaging system? Even if you hate R the language, R has among the most user-friendly, cross-plaform packaging systems I'm aware of.

https://stackoverflow.com/questions/10947159/writing-robust-...

Historically, the conventional way to write R code was one that tended to result in shadowed names (and hence brittle code).

Re: What's Next for R?

#53

Post by kgwgk » Sat, Dec 28, 2019, 11:56 AM UTC

pickdenis wrote: post #3

I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictabl…

From that link:

> A vector is what is called an array in all other programming languages except R

Vectors are called vectors in several "wispy" languages: Common Lisp, Scheme, Clojure...

> An array with two dimensions is (almost) the same as a matrix.

I think it's the same, not "almost" the same. At least in the current version of R:

  > class(array(1, c(2,3)))
  [1] "matrix"
  > identical(array(1, c(2,3)), matrix(1, nrow=2, ncol=3))
  [1] TRUE

In 4.0 there will be a change and the class of a matrix will be both "matrix" and "array", but I think the fact that there is no difference between a 2-dimensional array and a matrix remains.

Re: What's Next for R?

#54

Post by sammm » Sat, Dec 28, 2019, 12:36 PM UTC

I started at a company using Shiny for their applications and R as part of their data pipelines.

A huge pain point for us is the packaging system. It is absolutely awful. Packages constantly get overridden so we have to install packages in a specific order. Whenever I have reached out to the community (including prominent members, which have written R books) I have always been told to just use the latest version of all packages and just get on with it, which as anybody knows, isn’t always possible, especially as there are constantly breaking API changes.

I understand R’s history and that in general, it is a lot better than it use to be, but I would only recommend R is used for notebook style work and to keep it well away from production.

We have migrated to Python, which isn’t perfect, but the difference in logging and packaging has been night and day.

Re: What's Next for R?

#55

Post by glofish » Sat, Dec 28, 2019, 12:40 PM UTC

What's Next for R?

Doing the exact same thing we did before!

We have a new library called "dtplyr" (no seriously!) it is designed to save users from the arcane and obtuse sides of R by combining the power of "dplyr" and "data.table", the two libraries that were designed to save users from the arcane and obtuse sides of packages such as "data.frame" and ....

I wish I were kidding. There is the absurd contention in the R world that by introducing yet another weirdly named package people can avoid having to learn and suffer through the "real" R.

Re: What's Next for R?

#56

Post by truculent » Sat, Dec 28, 2019, 12:42 PM UTC

sammm wrote: post #54

I started at a company using Shiny for their applications and R as part of their data pipelines. A huge pain point for us is the packaging system. It is absolutely awful. Packages constantly get overridden so we have to install packages in a specific order. Whenever I have reached out to the community (including prominent members, which have written R books) I have always been told to just use the latest version of a…

I have also found R in production to be a nightmare. On packaging, the renv package seems to be the new way to try to manage things. It’s not perfect but seems to be a step up on what was around before. Have you tried it out at all?

Re: What's Next for R?

#57

Post by sammm » Sat, Dec 28, 2019, 1:33 PM UTC

truculent wrote: post #56

sammm wrote: post #54

I started at a company using Shiny for their applications and R as part of their data pipelines. A huge pain point for us is the packaging system. It is absolutely awful. Packages constantly get overridden so we have to install packages in a specific order. Whenever I have reached out to the community (including prominent members, which have written R books) I have always been told to just use the latest version of a…

I have also found R in production to be a nightmare. On packaging, the renv package seems to be the new way to try to manage things. It’s not perfect but seems to be a step up on what was around before. Have you tried it out at all?

I haven’t, thank you for the suggestion. I will give it a go.

Re: What's Next for R?

#58

Post by duckmysick » Sat, Dec 28, 2019, 1:37 PM UTC

nightski wrote: post #23

latte wrote: post #18

Cannot comment from my personal impressions, as I have almost zero knowledge of R, compared to several years of using Python for writing apps and working with data. I like R's focus on functional programming, though. However, a couple of years ago, my wife tried to transition from business consulting to a data analytics / data science role. She started with taking an R course. She was put off by R's complexity and th…

You only have to go through the learning process once. You are able to use the language for a lifetime. I find it so strange how much emphasis we tend to put on things being simple to learn and pick up.

Because if things aren't simple to learn and pick up, people will get discouraged and move on. As was the case in the comment above.

Great documentations and tutorials go a long way.

Re: What's Next for R?

#59

Post by amelius » Sat, Dec 28, 2019, 1:47 PM UTC

sammm wrote: post #54

I started at a company using Shiny for their applications and R as part of their data pipelines. A huge pain point for us is the packaging system. It is absolutely awful. Packages constantly get overridden so we have to install packages in a specific order. Whenever I have reached out to the community (including prominent members, which have written R books) I have always been told to just use the latest version of a…

Same experience here.

A lot of packages/functionality are not available in Python, however.

Re: What's Next for R?

#60

Post by PlugTunin » Sat, Dec 28, 2019, 3:29 PM UTC

curiousgal wrote: post #15

luhego wrote: post #10

I used R when I took an online course on Data Analysis. I didn't like it at all. Its syntax is weird and painful to read. The only nice things about R are Tidyverse and ggplot. I found Python to be a better alternative. You can use Pandas for data analysis y EDA. Matplotlib and Seaborn for plotting. Scikit-learn for training your models. An additional benefit is that Python is a general purpose language that you can…

In almost all of the use cases you mentionned, R blows Python out of the water. Working with dataframes in R is much much more convenient than Pandas (loc, iloc, etc??) Plotting is an obvious win for R, matplotlib is horrible, it's powerful yes but it is an absolute pain when compared to ggplot. Scikit is definitely unmatched but caret is not so far behind. Also, R has a plethora of implemented models that Python lac…

Exactly. I initially liked Pandas, but then I discovered what I can do with data frames in R, visualizations with ggplot, and the SQL-like data manipulation using dplyr w/ pipes from magrittr. R may have the steeper learning curve -- and for certain uses, be inferior to Python -- but it's a wonderful language.