Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A book to learn R and Python in parallel for Data Science

github.com

Post reply on HN
Print view View original

81–90 of 90 posts

Re: A book to learn R and Python in parallel for Data Science

#81

Post by Annatar »

psychometry wrote: post #58
Annatar wrote: post #42

Earlier quoted context omitted.

I maintain R and Shiny where I work. With Shiny, one can now build any web application imaginable, even a completely generic one. R has such a huge library of software now, that it has gone far and wide outside of statistics and analytics - any kind of application can be built in R now. In fact I see no point in using Python for mathematics or number crunching any more as R has it all and performance critical parts c…

Sorry to be so blunt, but if you think R is comparable with Python on the web app side, you haven't built any moderately complex web apps. Shiny is fine for single-page reactive apps, but it's not a generic framework in the way that Django or Flask is.

I hate web frameworks, absolutely despise those horrible, over-complicated, bloated monstrosities. If writing for "Flask" or "Django" is considered an advantage, then I'm deeply glad I didn't fall for that garbage.

Re: A book to learn R and Python in parallel for Data Science

#82

Post by psychometry »

Annatar wrote: post #81
psychometry wrote: post #58

Earlier quoted context omitted.

Sorry to be so blunt, but if you think R is comparable with Python on the web app side, you haven't built any moderately complex web apps. Shiny is fine for single-page reactive apps, but it's not a generic framework in the way that Django or Flask is.

I hate web frameworks, absolutely despise those horrible, over-complicated, bloated monstrosities. If writing for "Flask" or "Django" is considered an advantage, then I'm deeply glad I didn't fall for that garbage.

I suppose everyone using a web framework is just a rube, and that you alone have a monopoly on wisdom? Well, I take back my apology. Ignorance and arrogance together are not a good combination, but you possess both in spades.

As you gain experience building applications, at some point you'll learn that you were wrong. Or you won't, in which case I feel sorry for whoever inherits your reinvented-wheel codebase.

Re: A book to learn R and Python in parallel for Data Science

#83

Post by anthony_doan »

kuzehanka wrote: post #33
anthony_doan wrote: post #25

Earlier quoted context omitted.

> There's a bunch of comments below which can be summed up with 'use R because doesn't have a direct python equivalent' but they're all missing the point that the Python data science ecosystem is evolving at a much faster pace than R and will completely supersede it in a few years. The point is R is a very good language for statistic because of the packages not data science. Data science can do their own thing it's o…

Everything you said sums up with 'R is a very good language for statistic because of the packages' which is pretty much in agreement with the GP comment. R has nothing going for it except a rapidly dwindling number of packages that don't yet have a direct python equivalent. It doesn't make sense to invest time into R if one already knows python unless one specifically focusing on academia pure stats type stuff. Even…

> R has nothing going for it except a rapidly dwindling number of packages that don't yet have a direct python equivalent.

This is not a true statement.

Here are the data that goes against this statement.

1. https://www.r-bloggers.com/on-the-growth-of-cran-packages/ 2. https://blog.revolutionanalytics.com/2017/01/cran-10000.html 3. https://www.r-bloggers.com/rs-remarkable-growth/

From 2015 to 2016: ~6,200 to More than 8,000 in April, 2016

From 2016 to 2017: CRAN now has 10,000 R packages.

> Even then, the incoming generation of undergrads are increasingly proficient with programming and are shying away from R the same way that they shied away from Matlab after scipy matched it for 95% of their tasks.

This is a generalization.

So far you've made opinionated negative generalization with no data.

Python is great because it learn from Matlab and took many great ideas and inspirations from Matlab. But I'm not going to make sweeping negative statements about Matlab or pretend to know how it going when I don't have enough data or experiences in it.

Re: A book to learn R and Python in parallel for Data Science

#84

Post by snackematician »

kuzehanka wrote: post #76
snackematician wrote: post #63

Earlier quoted context omitted.

Tidyverse is not just some " " -- it's an entire workflow, centered around functional programming and tidy data ( https://vita.had.co.nz/papers/tidy-data.pdf ), and nothing in Python comes close. R has many warts, but its lisp roots and metaprogramming strengths have allowed the tidyverse devs, and other excellent programmers working with R, to dramatically improve the language, and spawn a whole new style of statist…

Can you elaborate on what tidyverse offers you that the python ecosystem doesn't? 'Nothing comes close' is a couple degrees too strong a statement from my experience with R, but maybe you know something I don't.

Tidyverse offers a programming style based around piping dataframes through a chain of endomorphisms ("verbs"). Closest things that come to mind are SQL and d3. Pandas feels clumsy by comparison.

Re: A book to learn R and Python in parallel for Data Science

#85

Post by Annatar »

psychometry wrote: post #82
Annatar wrote: post #81

Earlier quoted context omitted.

I hate web frameworks, absolutely despise those horrible, over-complicated, bloated monstrosities. If writing for "Flask" or "Django" is considered an advantage, then I'm deeply glad I didn't fall for that garbage.

I suppose everyone using a web framework is just a rube, and that you alone have a monopoly on wisdom? Well, I take back my apology. Ignorance and arrogance together are not a good combination, but you possess both in spades. As you gain experience building applications, at some point you'll learn that you were wrong. Or you won't, in which case I feel sorry for whoever inherits your reinvented-wheel codebase.

I won't, because I've been building applications for over 30 years; I know what's best, as I've had ample time to find out what works and what is sheer idiocy.

The various software I've written is small, fast, light, with minimal dependencies and it's easy to install because I deliver it as OS packages for the operating systems I support; my users tell me they are happy. The memory requirements are miniscule and the software lightning fast. Its size is measured in kilobytes, not megabytes or gigabytes, which means I must be doing something right. The manual pages often exceed the software in size and are brimming with examples. I pay extra attention to being backwards compatible when I implement changes and enhancements. Regressions are non-existent.

So I know I'm right and that using a bloated "framework" would have been one of the stupidest things I could have ever done.

As for re-inventing wheels, I use what comes with the OS and leverage what's already there; I've purposely not implemented any algorithm re-implementations of my own, although I easily could have. I'm neither dumb nor stupid to go re-inventing wheels, in fact that's one of the reasons why I hate webshits' frameworks. They don't call them webshit for no reason.

Re: A book to learn R and Python in parallel for Data Science

#86

Post by kuzehanka »

snackematician wrote: post #84
kuzehanka wrote: post #76

Earlier quoted context omitted.

Can you elaborate on what tidyverse offers you that the python ecosystem doesn't? 'Nothing comes close' is a couple degrees too strong a statement from my experience with R, but maybe you know something I don't.

Tidyverse offers a programming style based around piping dataframes through a chain of endomorphisms ("verbs"). Closest things that come to mind are SQL and d3. Pandas feels clumsy by comparison.

Uhhh but pandas is literally a chained architecture? Have you actually used it?

Re: A book to learn R and Python in parallel for Data Science

#87

Post by hdkrgr »

arendtio wrote: post #27
billfruit wrote: post #13

I sometimes wonder is there any reason to learn R at all, since python eco system has absorbed most of its advanced statistical functionality, coupled with the factor that python environment is much more general, with capabilities to fetch, decode/encoded data,work with binary data databases, web frameworks for presenting etc.

I don't know if it is still a thing, but if you are working with SAP HANA (in-memory database) there is a good chance you would like to learn R as they integrated it into their database.

Related: tidyverse's dbplyr let's you write tidyverse code querying almost any remote database - leveraging the DBs computation while writing code (almost) as you would for a local data frame. In my old job I got to a point where I would barely ever need to write sql anymore because of this. https://cran.r-project.org/web/packages/dbplyr/vignettes/dbp...

Re: A book to learn R and Python in parallel for Data Science

#88

Post by snackematician »

kuzehanka wrote: post #86
snackematician wrote: post #84

Earlier quoted context omitted.

Tidyverse offers a programming style based around piping dataframes through a chain of endomorphisms ("verbs"). Closest things that come to mind are SQL and d3. Pandas feels clumsy by comparison.

Uhhh but pandas is literally a chained architecture? Have you actually used it?

I have used pandas extensively, it was my main statistics environment for a couple years before I switched back to R for tidyverse. At the time chaining was not well supported or idiomatic; multi-indexing was all the rage.

I still occasionally use pandas with seaborn when it's not worth it to switch out to R. I don't think it can match the tidyverse+ggplot combo for quickly exploring and making beautiful plots. But this discussion has inspired me to do some googling and it seems like some people are using tidyverse-like workflows in pandas (https://stmorse.github.io/journal/tidyverse-style-pandas.htm...). Doesn't seem quite as smooth but I'll definitely be trying it out next time I'm working in pandas.

Re: A book to learn R and Python in parallel for Data Science

#89

Post by rpier001 »

tyingq wrote: post #40
hjk05 wrote: post #29

Earlier quoted context omitted.

What makes you think R or Python are bad at parallelism? My experience is that both are very decent.

Both have packages that can manage subprocesses. Both have inherently single threaded interpreters.

What technology in what language are you comparing them against?

Re: A book to learn R and Python in parallel for Data Science

#90

Post by piccolbo »

snackematician wrote: post #88
kuzehanka wrote: post #86

Earlier quoted context omitted.

Uhhh but pandas is literally a chained architecture? Have you actually used it?

I have used pandas extensively, it was my main statistics environment for a couple years before I switched back to R for tidyverse. At the time chaining was not well supported or idiomatic; multi-indexing was all the rage. I still occasionally use pandas with seaborn when it's not worth it to switch out to R. I don't think it can match the tidyverse+ggplot combo for quickly exploring and making beautiful plots. But t…

I've used both and have two additional comments.

Some of the dplyr elegance comes from the flexible evaluation mechanism in R, whereby mutate(data, col1+col2) works because the second arg is evaluated in an enriched environment. Python eschews this kind of macro-like extensions because, my guess, tampering with evaluation makes a lot of other things complicated (for instance, forget replacing args with their value, that doesn't work anymore). I think the author of dplyr himself in later work has promoted the use of the ~ operator to explicitly block eval of an argument and at least make these departures from regular eval explicit. That means dplyr is ahead for interactive use, but for programming you have to switch to a separate API (the underscore "verbs") and that makes the transition from interactive work to coding a bit steeper. It's all trade-offs, and I am not saying that I know better than either the pandas or dplyr authors.

As to ggplot, if you believe the future of statistical graphics is in-browser and interactive, you should take a look at altair for python (I myself created a small extension to it called altair_recipes). It's based on vega, like ggplot anointed (but not quite ready) successor ggvis and uses the grammar of graphics (or on interpretation thereof) like ggplot, with extensions to interaction. Simpler than D3 by most accounts.

Post reply on HN