Live data from Hacker News

A book to learn R and Python in parallel for Data Science

github.com

61–70 of 90 posts

Re: A book to learn R and Python in parallel for Data Science

#61
post #46

Earlier quoted context omitted.

I cannot understand why I would use Python over R. R is designed from the ground up for massive amounts of data processing at speed and with ease. Even if Python continues accreting computational functionality, it will never be as fast or as efficient as R. Improving Python for something R is designed to do seems to me to be a huge waste of time: familiarity should not be the driving force behind replicating R's func…

> R is designed from the ground up for massive amounts of data processing at speed What? The R ecosystem doesn't provide meaningful out of core capabilities, nevermind the ability to handle anything approaching 'massive amounts of data'. -- Would sure love to know why an agenda-less factual comment is getting downvoted.

In my experience, R is really fast since I t was designed to store data in columnar format which we now all know is best for data analysis. So, in most cases, scaling up computation is quite easy. To scale out, you can use Apache Spark with R, the interface I’ve worked on, sparklyr is quite easy to use and allows you to scale out computation. Just to give you an example of what’s possible, I was playing around yesterday with a ray tracing prototype someone is building and scaled it out in Spark, see https://twitter.com/javierluraschi/status/112055769372135424... — it’s a misconception that R is slow or can’t scale.

Re: A book to learn R and Python in parallel for Data Science

#62

Earlier quoted context omitted.

> R is designed from the ground up for massive amounts of data processing at speed What? The R ecosystem doesn't provide meaningful out of core capabilities, nevermind the ability to handle anything approaching 'massive amounts of data'. -- Would sure love to know why an agenda-less factual comment is getting downvoted.

In my experience, R is really fast since I t was designed to store data in columnar format which we now all know is best for data analysis. So, in most cases, scaling up computation is quite easy. To scale out, you can use Apache Spark with R, the interface I’ve worked on, sparklyr is quite easy to use and allows you to scale out computation. Just to give you an example of what’s possible, I was playing around yester…

You can plug any compute kernel you want into spark, that's not a pro or con of R.

Column stores are standard in any analytics pipeline today. They make up Python's Pandas, R's dplyr, and Java's DataFrame. How or why does R stand out for 'massive amounts of data'?

R does not have have meaningful out of core compute offerings that compare with something like Dask.

R does not at all have cluster compute offerings that compare to Dask Distributed.

If you want to know what real performance looks like, check out Python's cudf which will shortly fully match the Pandas api. That raytracing example you linked would run at interactive rates with cudf, I really don't see any basis for perf arguments in R's favour, and 'massive data' arguments are laughable here.

Whatever advantages R has, perf or scalability are definitely not amongst them.

Re: A book to learn R and Python in parallel for Data Science

#63

I sometimes wonder is there any reason to learn R at all, since python eco system has absorbed most of its advanced statistical functionality, coupled with the factor that python environment is much more general, with capabilities to fetch, decode/encoded data,work with binary data databases, web frameworks for presenting etc.

I don't think there's any reason to learn R for anyone who is already proficient at programming. Despite being proficient with R, the only times I used it in the last two years were for ggplot. And even for data vis, I'm increasingly using Python and JS. There's a bunch of comments below which can be summed up with 'use R because doesn't have a direct python equivalent' but they're all missing the point that the Pyth…

Tidyverse is not just some "" -- it's an entire workflow, centered around functional programming and tidy data (https://vita.had.co.nz/papers/tidy-data.pdf), and nothing in Python comes close. R has many warts, but its lisp roots and metaprogramming strengths have allowed the tidyverse devs, and other excellent programmers working with R, to dramatically improve the language, and spawn a whole new style of statistical programming.

Re: A book to learn R and Python in parallel for Data Science

#64

Python and Julia might make more sense today.

Is Julia actually used that much? I've been hearing people herald it as the next big thing for the last five years or so, but it doesn't seem like it has taken off. I personally don't know anybody who uses it professionally (I know plenty of people who use R professionally). The most recent SO survey also indicates that it is rather unpopular.

Re: A book to learn R and Python in parallel for Data Science

#65
Apart from the odd library I have rarely found much benefit to using both languages for DS as you end up expressing the same paradigms just in different syntax. And I think for good reason too - the basis of the tools used to do data science arent in the languages themselves but the packages built for the task which is why there's often an R equivalent of a Python package and vice versa. So in effect almost no one 'uses' R/Python for DS as much as Rube-Goldberg highly-optimised compiled libraries together using different syntax.ie dplyr/pandas/scipy/ggplot etc are the real stars of the show.

Rather than R vs Python I hope one of two things happen. Either both languages get replaced by a 'better' ML language eg Swift / Julia giving us users a 'turtles all the way down' experience and removing the reliance on complied packages. Or, second option, they get relegated even further into being nothing but glue between some common data formats specific to the type of work found in DS allowing you the user basically a choice between syntactic-sugar of one glue-language versus the other. Something like Apache Arrow springs to mind but I'm not sure where they are at the moment

Re: A book to learn R and Python in parallel for Data Science

#66
post #32

R has a nice web development framework called Shiny. While it is not comparable to say Django or Flask, Shiny does make it incredibly easy to share data analysis. If one wants to share statistical analysis or create a data oriented dashboard, then there is definitely a reason to consider R and Shiny. Note that Python has Dash, which is comparable to Shiny, but it is less mature as far as I know. While previously Shin…

Shiny is fantastic! (Especially paired up with RStudio Connect)

Re: A book to learn R and Python in parallel for Data Science

#67
post #49

Earlier quoted context omitted.

I use both Python and R. tidyverse/ggplot2 alone are enough reason to use R, and are substantially faster for tasks that utilize those packages than the equivalent in Python (in my opinion). Although I haven't had as much reason to use base R. For more ML-related tasks I do go back to Python.

Here here. Tidyverse also provides a centralised 'this is how you do X' nexus really helps discover-ability. World class stuff, on tap. For example, I know the recommended pipe in R is magrittr's %>%. I have no idea what the respectable pipe library in Python is, or even if there is one. I wouldn't even know where to start finding all the tidyverse equivalents in Python. It isn't as organised and obvious as the R sta…

Julia has a |> operator and it works amazingly well with Queryverse.jl which is a clone of Tidyverse!

Re: A book to learn R and Python in parallel for Data Science

#68
post #60

Shouldn't '<-' be used instead of '=' for variable assignments, as they aren't the same thing in R.

It's a large source of bike-shedding in the R community but out of the 5 assignment operators in R, those two are largely the same.

There's a good explanation here: https://stackoverflow.com/questions/1741820/what-are-the-dif...

Re: A book to learn R and Python in parallel for Data Science

#69
post #60

Shouldn't '<-' be used instead of '=' for variable assignments, as they aren't the same thing in R.

They are the same thing, minus the corner case of assignment within a function call:

e.g.

  divide = function(x, y) {
    return(x/y)
  }

  divide(y = 2, x = 1)
  divide(y 
These two calls give the same result, as the second results in assignment and then passing the argument by position. Other than this case, they are exactly interchangeable.

Re: A book to learn R and Python in parallel for Data Science

#70
post #64

Python and Julia might make more sense today.

Is Julia actually used that much? I've been hearing people herald it as the next big thing for the last five years or so, but it doesn't seem like it has taken off. I personally don't know anybody who uses it professionally (I know plenty of people who use R professionally). The most recent SO survey also indicates that it is rather unpopular.

Julia just got to 1.0 last year, and it does have areas where it's already between the best options in scientific computing such as differential equations solving and mathematical optimization. Regardless of not being the most popular (against the behemoths that have many times it's age and support), you shouldn't have trouble doing most stuff with it from machine learning to statistics. And it's a pretty fun and fairly unique language to learn and use.
Post reply on HN