Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's Next for R?

qz.com

Post reply on HN
Print view View original

41–50 of 72 posts

Re: What's Next for R?

#41

Post by bsg75 »

zosima wrote: post #8
pickdenis wrote: post #3

I know this is a dead horse, but I think R seriously shot itself in the foot with its data structures[1]. I don't really see a solution for this, as fixing it would never be backward compatible. I'll always pick Python over R because the data structures actually make sense to me as a programmer (objects that look like lists, dicts, matrices, etc. or any combination of the above, and they all behave in very predictabl…

True, the default semantics of R's data structures are somewhat arcane (of course as they're based on S [1] from the 70's). And the current support for e.g. 64bit integers leaves something to be desired. But behind the scenes, R is just a lisp with some data structures that are adapted to statistics and data science. All base data structures are by default immutable. And e.g. the vector type is extremely performant a…

> 64bit integers leaves something to be desired

This is something I wish there was more progress on. A serious limitation in some contexts.

Re: What's Next for R?

#42

Post by CoolGuySteve »

609venezia wrote: post #27
meztez wrote: post #11

I would highly recommend the use of the package data.table over tibble or the basic data.frame if you are doing any type of modeling in R with larger datasets. Yes R has many data structures but knowing how to use data.table will blow your mind in term of efficiency. Matt and other contributors have built something extremely fast and flexible. I get that R is not for everyone but used correctly it is a beast. Now thi…

> I believe Julia to be the future but so far the adoption rate in house has been low. Why do you believe it will be the future, and what do you see as the barriers to roll-out? I ask as someone who is curious about when/whether to start investing in Julia competence

Both R and pandas force you to wrap your problem around dataframes and vectorized operations. But sometimes you really do just want to write a loop that iterates over the data.

Right now the only way to do that without significant performance costs is to drop down into C or avoid the problem completely by using Julia.

Having worked with both R and Python on large datasets, I think both languages are really easy until they aren’t. Eventually you hit a performance wall.

Re: What's Next for R?

#43

Post by bransonf »

Disappointed in the lack of discussion of R-Shiny or Plumber.

R-Shiny is a full stack platform for web apps, and it’s how I leveraged my data science background to get into web development. It’s incredibly powerful in my opinion, with the only obvious limitation being the speed of R itself.

And Plumber. It’s become the defacto method for deploying R code in a REST api. It too is still maturing, but I see it eventually becoming the Flask of R.

Truth be told, however, after developing quite a few projects on the Shiny/Plumber stack, I wouldn’t recommend anyone do it.

If for some reason you can only have an R interpreter, go for it. But learning multiple languages really is the best solution if you want to manage efficient applications. I say this, however, realizing that all of my colleagues writing R don’t have engineering backgrounds.

I can’t help but feel like R is like JavaScript in many ways. Ease of use and the ease of publishing packages very quickly clutters the repository.

R will always have a special place in my heart, after all it’s the language that made me discover programming. However, I can’t help but feel that my thirst for efficiency is making me outgrow it as a language quickly.

Re: What's Next for R?

#44

Post by goatlover »

CoolGuySteve wrote: post #42
609venezia wrote: post #27

Earlier quoted context omitted.

> I believe Julia to be the future but so far the adoption rate in house has been low. Why do you believe it will be the future, and what do you see as the barriers to roll-out? I ask as someone who is curious about when/whether to start investing in Julia competence

Both R and pandas force you to wrap your problem around dataframes and vectorized operations. But sometimes you really do just want to write a loop that iterates over the data. Right now the only way to do that without significant performance costs is to drop down into C or avoid the problem completely by using Julia. Having worked with both R and Python on large datasets, I think both languages are really easy until…

You can drop down into the Numpy values array in Pandas to get your performance gain when iteration is otherwise slow.

Re: What's Next for R?

#45

Post by ACow_Adonis »

meztez wrote: post #37
ACow_Adonis wrote: post #36

Earlier quoted context omitted.

As someone "fully fluent" in both, for many workflows that can be properly implemented in SAS, you would expect on a technical level the SAS program could be faster. It's a fully compiled language, it's a "simple" compilation model (compared to R), and the interaction between incremental compilation and the macro system allows you to do some really good blurring between run-time and compilation when performance matte…

Believe me, I know. The code just becomes unreadable when you put all execution inside the same data step and use hash table to do fast small to big merging. And not to mention debugging that mess when you have a macro layer on top of it. Not having access to function source code, installation process being what it was. I do not miss it. And yes technically SAS is faster than R but part of the equation is how many pe…

I don't think that's being emotional at all.

I'm being emotional when I say I have a soft spot for it because of some nostalgia and occasionally dropping in to do some "rock star" programming moments with it. But that's the opposite of what I'd want if/when I was running my own ship.

I too almost always try to steer myself and others away from it now because of the licensing/customer hostility. It's absolutely ridiculous...

Re: What's Next for R?

#46

Post by ACow_Adonis »

jrumbut wrote: post #39
ACow_Adonis wrote: post #36

Earlier quoted context omitted.

As someone "fully fluent" in both, for many workflows that can be properly implemented in SAS, you would expect on a technical level the SAS program could be faster. It's a fully compiled language, it's a "simple" compilation model (compared to R), and the interaction between incremental compilation and the macro system allows you to do some really good blurring between run-time and compilation when performance matte…

Do you have any resources that help explain these SAS performance measures? A book perhaps? I have been trying to help with exactly this (and your breadcrumbs help) but it is tricky for me since I am used to open source/*nix environment where you can use much different tools and also information and tutorials are distributed much more widely.

Unfortunately not. With SAS I never used books and relied solely on having access to the fully licensed system at a previous job and all of the SAS PDFs floating around the internet and findable with specific searches.

That combined with a general computer science background and you can start to put the whole thing together.

I'd be lying if I said I hadn't considered writing one, but at my age I'd honestly ask why write one for an old proprietary system and make business for someone else when, if I'll ever go back long term, they can pay me an exorbitant amount as a consultant. Might as well start writing 'the dark arts of COBOL' :p

Re: What's Next for R?

#47

Post by amrrs »

CoolGuySteve wrote: post #42
609venezia wrote: post #27

Earlier quoted context omitted.

> I believe Julia to be the future but so far the adoption rate in house has been low. Why do you believe it will be the future, and what do you see as the barriers to roll-out? I ask as someone who is curious about when/whether to start investing in Julia competence

Both R and pandas force you to wrap your problem around dataframes and vectorized operations. But sometimes you really do just want to write a loop that iterates over the data. Right now the only way to do that without significant performance costs is to drop down into C or avoid the problem completely by using Julia. Having worked with both R and Python on large datasets, I think both languages are really easy until…

You can increase the speed of loops in Python using Numba. It's really a great performance booster with just a few decoraters added.

Re: What's Next for R?

#48

Post by lottin »

latte wrote: post #18

Cannot comment from my personal impressions, as I have almost zero knowledge of R, compared to several years of using Python for writing apps and working with data. I like R's focus on functional programming, though. However, a couple of years ago, my wife tried to transition from business consulting to a data analytics / data science role. She started with taking an R course. She was put off by R's complexity and th…

R has a number of features that are intended to facilitate interactive use, which despite being very convenient can be confusing to someone who is trying to learn the language. With Python, on the other hand, it is easier for a novice to build a mental model of how things work. However Python is pretty awful as an interactive language due to the way it interprets white space. Personally I think taking the time to learn R is well worth it.

Re: What's Next for R?

#49

Post by bllguo »

bransonf wrote: post #43

Disappointed in the lack of discussion of R-Shiny or Plumber. R-Shiny is a full stack platform for web apps, and it’s how I leveraged my data science background to get into web development. It’s incredibly powerful in my opinion, with the only obvious limitation being the speed of R itself. And Plumber. It’s become the defacto method for deploying R code in a REST api. It too is still maturing, but I see it eventuall…

on the shiny note - check out streamlit. declarative python equivalent. it's pretty incredible how easy it is to use

Re: What's Next for R?

#50

Post by whoisnnamdi »

curiousgal wrote: post #15
luhego wrote: post #10

I used R when I took an online course on Data Analysis. I didn't like it at all. Its syntax is weird and painful to read. The only nice things about R are Tidyverse and ggplot. I found Python to be a better alternative. You can use Pandas for data analysis y EDA. Matplotlib and Seaborn for plotting. Scikit-learn for training your models. An additional benefit is that Python is a general purpose language that you can…

In almost all of the use cases you mentionned, R blows Python out of the water. Working with dataframes in R is much much more convenient than Pandas (loc, iloc, etc??) Plotting is an obvious win for R, matplotlib is horrible, it's powerful yes but it is an absolute pain when compared to ggplot. Scikit is definitely unmatched but caret is not so far behind. Also, R has a plethora of implemented models that Python lac…

Just as an FYI - the statsmodels python package just released numerous new time series tools in version 0.11 rc1 [1] and also has functions for quantile regression [2]

[1] https://github.com/statsmodels/statsmodels/releases [2] https://www.statsmodels.org/dev/examples/notebooks/generated...

Post reply on HN