Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Another Book on Data Science – Learn R and Python in Parallel

anotherbookondatascience.com

Post reply on HN
Print view View original

71–80 of 96 posts

Re: Another Book on Data Science – Learn R and Python in Parallel

#71

Post by KKPMW »

clarityPhone wrote: post #2

I skimmed through the book, and think it does a very poor job at showcasing how R and Python are juxtaposed in industry. To be fair, the book advertises showing R and Python code side-by-side. And that’s what it does. But it does it unlike how the languages are most often used in industry. As a quick example, I saw no tidyverse code, which is essentially the only thing keeping R in the game. Learning R from this book…

> tidyverse code, which is essentially the only thing keeping R in the game

From my experience this is not the case. In biomedicine and bioinformatics few people actually use tidyverse because the data is much better represented as a matrix, and not in the "tidy" form.

Outside of that corporations (well at least 2 I contracted with) used `data.table` explicitly. Join 3 ad-click dataframes matching by userID, sessionID and closest possible time-point - that's one line in `data.table`.

Tidyverse is well suited for learning and for managing (relatively) simple datasets. But becomes cumbersome for more complex data. It can be used for those data too of course, just that it will be adding ad-hoc solutions and maybe get in a way more than help.

Re: Another Book on Data Science – Learn R and Python in Parallel

#72

Post by RosanaAnaDana »

thomasfedb wrote: post #9
clarityPhone wrote: post #2

I skimmed through the book, and think it does a very poor job at showcasing how R and Python are juxtaposed in industry. To be fair, the book advertises showing R and Python code side-by-side. And that’s what it does. But it does it unlike how the languages are most often used in industry. As a quick example, I saw no tidyverse code, which is essentially the only thing keeping R in the game. Learning R from this book…

Is tidyverse really the only option? I'm a big fan of data.table + magrittr as a very powerful data munging combo.

This is what I've got my crew running. Tidyverse is basicly worthless once you hit a certain scale of data. If you've got datatables representing.

List functionality within datatable is blazingly fast. Faster than anything else I've seen in python or R.

Re: Another Book on Data Science – Learn R and Python in Parallel

#73

Post by adamnemecek »

krapht wrote: post #42
adamnemecek wrote: post #18

Earlier quoted context omitted.

What is there besides semantics?

Uh, adoption by the community at large? The best program is the one you didn't have to write because a package existed for it already. Just because a language has a foreign function interface doesn't mean it's easy to interact with other libraries. R, Python, Matlab, and C++ are the big dogs in scientific programming, and the inertia behind having a large community behind than will continue to drive adoption.

That's my point, Julia's interop is unparalleled. Like it legit takes one line to call a python library. You get back a result in native julia type.

Re: Another Book on Data Science – Learn R and Python in Parallel

#74

Post by RosanaAnaDana »

_2d30 wrote: post #54
notafraudster wrote: post #35

Earlier quoted context omitted.

magrittr is part of the tidyverse, but I agree that data.table is a comparably powerful and sometimes faster option versus dplyr.

Almost always faster, actually. https://github.com/Rdatatable/data.table/wiki/Benchmarks-:-G...

I was going to make this point, but yeah. The only thing I think people have a bit of a time with is how you do operations in data.table. If you are coming from plyr/dplyr, the transition can be difficult. However, I've found that the more I do, the more I prefer it, inspite of the fact that the main reason I use dt over tidy is the phenomenal performance gain.

Re: Another Book on Data Science – Learn R and Python in Parallel

#75

Post by RosanaAnaDana »

thomasfedb wrote: post #56
Bootvis wrote: post #38

Earlier quoted context omitted.

magrittr existed before the tidyverse and can be used on standalone perfectly fine. In all benchmarks I've seen data.table is faster than dplyr on all tasks. Curious to see other results.

At the scale of what I'm doing the benchmarks don't sway me, but I do like the syntax of data.table - it feels a bit like relational algebra.

So then I would assume you must be working with tables of less than 1000 rows, because thats pretty much the only case where it doesn't matter. At anything more than 1k rows, the differences are substantial.

Re: Another Book on Data Science – Learn R and Python in Parallel

#76

Post by randomvectors »

RosanaAnaDana wrote: post #70
randomvectors wrote: post #60

Earlier quoted context omitted.

> built-in dataframe support Not an advantage if you ask me - exactly because data.frame is built in, people have been building their own versions (tibble, data.table) instead of improving it. That's how R ended up with 3 different structures that are similar but have inconsistent apis and behaviour. > lots of domain-specific packages That's true. > more consistent interfaces for basic statistics and machine learning…

You mean like keras? or tensorflow? Or base random forest. You know, like the original Breiman implementation. Python has utility. But R is far superior in its the quality of the packages, their documentation, their ability to behave predictably on a given data type. I run a machine learning shop. Right now all of the training, application, and data management is handled via R. R is simply superior in too many ways f…

> You mean like keras? or tensorflow? Or base random forest. You know, like the original Breiman implementation.

> ...

> Since we're moving some big applications to keras/ TF we do use python and will be using more in the future.

Not sure if I misunderstood, or you're contradicting yourself there.

> R is far superior in its the quality of the packages, their documentation, their ability to behave predictably on a given data type.

I not only disagree but I think that the exact opposite is true for each one of these points. But if things are working well in our shop, I'm not going to try to convince you otherwise.

Re: Another Book on Data Science – Learn R and Python in Parallel

#77

Post by lottin »

claytonjy wrote: post #67
kgwgk wrote: post #40

Earlier quoted context omitted.

> Python is definitely better for matrices (because of NumPy) How so?

numpy is significantly faster and arguably more usable (e.g. broadcasting) than anything in R, and only recently has there been progress in more efficient matrix manipulation in R like rray[0], a wrapper for xtensor. [0] https://github.com/r-lib/rray

As far as I can tell, R uses BLAS for matrix operations, and Python probably does the same, so in terms of efficiency I wouldn't expect a big difference between the two.

Re: Another Book on Data Science – Learn R and Python in Parallel

#78

Post by RosanaAnaDana »

randomvectors wrote: post #76
RosanaAnaDana wrote: post #70

Earlier quoted context omitted.

You mean like keras? or tensorflow? Or base random forest. You know, like the original Breiman implementation. Python has utility. But R is far superior in its the quality of the packages, their documentation, their ability to behave predictably on a given data type. I run a machine learning shop. Right now all of the training, application, and data management is handled via R. R is simply superior in too many ways f…

> You mean like keras? or tensorflow? Or base random forest. You know, like the original Breiman implementation. > ... > Since we're moving some big applications to keras/ TF we do use python and will be using more in the future. Not sure if I misunderstood, or you're contradicting yourself there. > R is far superior in its the quality of the packages, their documentation, their ability to behave predictably on a giv…

My point behind the keras/ TF comment is that the libraries have front ends in both python and R, so its mix mox/ dealers choice on what you like to work in (since the backends of both are identical).

The primary reason to moving these to python is due to convenience/ the community. Most new work is published in python. If we find a new/ interesting model we want to implement, its probably written in python. Rather than reskin the thing in its entirety, its easier here to work in python.

A couple disclaimers: my group works primarily in geospatial data, and principally in LiDAR and multispectral imagery.

The coarse division I see between R/ Python, is that if you come from a research/ academic background (non-engineering), you probably learned to program in R. If you were an engineer, you probably learned matlab. If you are self taught/ coursera/ youtube, you probably learned in python.

R libraries are generally more geared towards academic research, and specifically, working within existing frameworks (handling geospatial data as geospatial data rather then turning them into a numpy arrays). Working in python, there is far more re-invention of the wheel, and its always a pain the ass to get things back into the structures they came in as.

Python has huge utility and is an important tool for certain work. But its really really not faster than R (it def used to be, this isnt the case any more).

R has better support for more scientific programming than python.

Re: Another Book on Data Science – Learn R and Python in Parallel

#79

Post by olooney »

lottin wrote: post #77
claytonjy wrote: post #67

Earlier quoted context omitted.

numpy is significantly faster and arguably more usable (e.g. broadcasting) than anything in R, and only recently has there been progress in more efficient matrix manipulation in R like rray[0], a wrapper for xtensor. [0] https://github.com/r-lib/rray

As far as I can tell, R uses BLAS for matrix operations, and Python probably does the same, so in terms of efficiency I wouldn't expect a big difference between the two.

Both R and numpy use BLAS, and if both are linked to the same library, say OpenBLAS or Intel MKL, then performance is in fact almost identical for expensive operations like matrix multiplication. (R also ships with its own internal BLAS implementation, which is reliable but not very fast, and I believe is still single threaded, so the first thing you should do if you are using R and care about performance is to swap it out.)

For more sophisticated linear algebra algorithm, such as SVD, both will use typically LAPACK, and again, both will exhibit essential identical performance.

There is one important difference though: when R is compiled for 64-bit machines, it can only use 64-bit floats! While numpy can support 32 and even (through software emulation) 16 bit floats. This can halve memory usage, which in turn halves cache misses, which results in a significant speed up in cases where 64-bits of precision is not needed.

Re: Another Book on Data Science – Learn R and Python in Parallel

#80

Post by zelda_1 »

KKPMW wrote: post #71
clarityPhone wrote: post #2

I skimmed through the book, and think it does a very poor job at showcasing how R and Python are juxtaposed in industry. To be fair, the book advertises showing R and Python code side-by-side. And that’s what it does. But it does it unlike how the languages are most often used in industry. As a quick example, I saw no tidyverse code, which is essentially the only thing keeping R in the game. Learning R from this book…

> tidyverse code, which is essentially the only thing keeping R in the game From my experience this is not the case. In biomedicine and bioinformatics few people actually use tidyverse because the data is much better represented as a matrix, and not in the "tidy" form. Outside of that corporations (well at least 2 I contracted with) used `data.table` explicitly. Join 3 ad-click dataframes matching by userID, sessionI…

I have a similar feeling. And that is why I spent one whole chapter in data.table (and pandas). Hope more R users would like to learn and use data.table.
Post reply on HN