Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's Next for R?

qz.com

Post reply on HN
Print view View original

11–20 of 72 posts

Re: What's Next for R?

#11

Post by meztez »

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible.

I get that R is not for everyone but used correctly it is a beast.

Now this is anecdotal, but we have in the insurance industry what we call on level premium calculators. It is basically a program that will rerate all policies with the current set of rates.

Our current R program can rate 41000 policies a second fully vectorized on a user laptop that has a an i5 from 2015.

In contrast, the previous SAS program could do 231 policies a minute on xeon 64 core processor from 2017.

For our workload and type of work, R has been a godsend.

Bonus, we can put what our data scientist develop in R directly in production. (after peer review, testing, etc, not different than any other production code)

Back when I started in 2005, we modeled in some proprietary software like Emblem, used Excel to build a first draft premium calculator, rebuilt the computation in SAS for the onlevel program and sent specs to IT to rebuilt the program again for production. All three had to produce the same results.

I've tried Python, Go, Rust, Julia. I'd say Python could be a good alternative but speed of data.table, RStudio IDE and ease of package management in R makes R an obvious choice for us. I believe Julia to be the future but so far the adoption rate in house has been low.

Re: What's Next for R?

#12

Post by xiaodai »

meztez wrote: post #11

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible. I get that R is not for everyone but used correctly it is a beast. Now thi…

For larger-than-RAM data I would recommend diskframe.com

It uses dplyr and data.table syntax to manipulate data on disk

Re: What's Next for R?

#13

Post by meztez »

xiaodai wrote: post #12
meztez wrote: post #11

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible. I get that R is not for everyone but used correctly it is a beast. Now thi…

For larger-than-RAM data I would recommend diskframe.com It uses dplyr and data.table syntax to manipulate data on disk

Thanks, so far we just scaled up our vm ram but i might find a use for it.

Re: What's Next for R?

#14

Post by anthony_doan »

downerending wrote: post #4
pickdenis wrote: post #3

I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictabl…

It's based pretty directly on S, which was designed in the mid 1970s. Yeah, it has very rough edges here, but hard to argue that they should have foreseen the future back then. That said, the real value in R seems to be the libraries. Has anyone looked at a shim that could make those libraries available to Python in a reasonably natural way? If that existed, the R language itself could be allowed to finally rest in p…

There is something to be said to build a programing language to solve a certain task in mind.

Being vector aware and having a dataframe support in R is much more elegant for me than Python's add on library. It's like Scala building on top of Java but trying to have an Actor paradigm vs Erlang built from get go around concurrency and choosing Actor as it's main concurrency paradigm. You can see this in other language on PHP and C++ let you be OOP but it's an after thought compare to Ruby or Python.

Re: What's Next for R?

#15

Post by curiousgal »

luhego wrote: post #10

I used R when I took an online course on Data Analysis. I didn't like it at all. Its syntax is weird and painful to read. The only nice things about R are Tidyverse and ggplot. I found Python to be a better alternative. You can use Pandas for data analysis y EDA. Matplotlib and Seaborn for plotting. Scikit-learn for training your models. An additional benefit is that Python is a general purpose language that you can…

In almost all of the use cases you mentionned, R blows Python out of the water.

Working with dataframes in R is much much more convenient than Pandas (loc, iloc, etc??)

Plotting is an obvious win for R, matplotlib is horrible, it's powerful yes but it is an absolute pain when compared to ggplot.

Scikit is definitely unmatched but caret is not so far behind. Also, R has a plethora of implemented models that Python lacks (from something as basic as decent quantile regression to time series analysis tools).

As for building a complete application, Python is indeed the go-to.

Syntax wise, using magrittr's pipes is an absolute pleasure. Good luck doing that with Python.

Re: What's Next for R?

#16

Post by gdevenyi »

curiousgal wrote: post #15
luhego wrote: post #10

I used R when I took an online course on Data Analysis. I didn't like it at all. Its syntax is weird and painful to read. The only nice things about R are Tidyverse and ggplot. I found Python to be a better alternative. You can use Pandas for data analysis y EDA. Matplotlib and Seaborn for plotting. Scikit-learn for training your models. An additional benefit is that Python is a general purpose language that you can…

In almost all of the use cases you mentionned, R blows Python out of the water. Working with dataframes in R is much much more convenient than Pandas (loc, iloc, etc??) Plotting is an obvious win for R, matplotlib is horrible, it's powerful yes but it is an absolute pain when compared to ggplot. Scikit is definitely unmatched but caret is not so far behind. Also, R has a plethora of implemented models that Python lac…

His #1 requirement was not being a painful language and nothing but being not-R can resolve that.

I use R everyday for statistical analysis due to it having certain interfaces and I still hate it every day.

Re: What's Next for R?

#17

Post by cwyers »

downerending wrote: post #4
pickdenis wrote: post #3

I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictabl…

It's based pretty directly on S, which was designed in the mid 1970s. Yeah, it has very rough edges here, but hard to argue that they should have foreseen the future back then. That said, the real value in R seems to be the libraries. Has anyone looked at a shim that could make those libraries available to Python in a reasonably natural way? If that existed, the R language itself could be allowed to finally rest in p…

There are various ways to call R from Python, or Python from R. They never end up being very idiomatic, which typically makes them a pain to work with.

Re: What's Next for R?

#18

Post by latte »

Cannot comment from my personal impressions, as I have almost zero knowledge of R, compared to several years of using Python for writing apps and working with data. I like R's focus on functional programming, though.

However, a couple of years ago, my wife tried to transition from business consulting to a data analytics / data science role. She started with taking an R course. She was put off by R's complexity and the course's early focus on the details of R syntax, function definitions, closures etc. and abandoned it.

The year after, she decided to try again and enrolled in a course that used Python (with numpy+pandas+scipy as data science stack) and she reported it to be much simpler, more intuitive and easier to learn compared to her previous experience with R. Now she has successfully completed the program and is employed as a data analyst.

Re: What's Next for R?

#19

Post by petulla »

meztez wrote: post #11

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible. I get that R is not for everyone but used correctly it is a beast. Now thi…

This may be useful. I prefer dplyr's syntax. https://github.com/tidyverse/dtplyr

Re: What's Next for R?

#20

Post by bachmeier »

downerending wrote: post #4
pickdenis wrote: post #3

I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictabl…

It's based pretty directly on S, which was designed in the mid 1970s. Yeah, it has very rough edges here, but hard to argue that they should have foreseen the future back then. That said, the real value in R seems to be the libraries. Has anyone looked at a shim that could make those libraries available to Python in a reasonably natural way? If that existed, the R language itself could be allowed to finally rest in p…

You can embed an R interpreter in any language with a C interface. That said, most of the complaints I see about R reflect preferences and prior programming experience with newer programming languages. While there are things I don't like about R, it's a Scheme without s-expressions, and overall I like it.
Post reply on HN