Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R and pandas and what I've learned about each

blog.yhathq.com

Post reply on HN
Print view View original

11–20 of 24 posts

Re: R and pandas and what I've learned about each

#11

Post by teoruiz »

To install the whole set of Python modules needed and iPython in a virtualenv (trick: there is no "pylab" module to install):

  % virtualenv --distribute --no-site-packages pandas_venv
  [blahblah]
  % . pandas_venv/bin/activate
  (pandas_venv) % easy_install readline # Probably only needed in Mac OS X for iPython to behave 
  [blahblah]
  (pandas_venv) % pip install ipython
  [blah blah]
  (pandas_venv) % pip install numpy
  [lots of blahblah]
  (pandas_venv) % pip install matplotlib
  [quite a bit of blahblah]
  (pandas_venv) % pip install pandas
  [some more blah blah]
  (pandas_venv) % pandas_venv/bin/ipython --no-banner
  
  In [1]: import numpy as np
  
  In [2]: import pandas as pd
  
  In [3]: import pylab as pl
  
  In [4]:

Re: R and pandas and what I've learned about each

#13

Post by oddthink »

wesm wrote: post #10
crayola wrote: post #2

Interesting analysis, but it would really benefit from a section about data.table. For me and many others, data.table has almost completely replaced data.frame (of which data.table is a subclass) and completely replaced plyr. The speed and ease of use of data.table are much more favorably comparable with pandas than the R tools mentioned here.

Comparisons with data.table on performance are much more favorable than with vanilla R or plyr; a lot of progress has been made last couple years, too. I personally find the data.table syntax to be a bit obtuse at times but it's a great library.

Aside from the performance differences, data.table makes it very easy to do interactive manipulation, at the cost of making it hard to program. Pandas currently goes in the opposite direction.

I'd rather have R/data.table at the prompt and python/pandas in my script, but if you have to err on one side, the python/pandas "low magic" is the side to err on. Pandas does have its own strange corners, though. For example, it seems like it tries hard to stick similar-typed columns into contiguous matrices, which leads to some unexpected casting, and I have no idea what the supposed benefit is over just keeping distinct columns.

Re: R and pandas and what I've learned about each

#15

Post by jofer »

teoruiz wrote: post #11

To install the whole set of Python modules needed and iPython in a virtualenv (trick: there is no "pylab" module to install): % virtualenv --distribute --no-site-packages pandas_venv [blahblah] % . pandas_venv/bin/activate (pandas_venv) % easy_install readline # Probably only needed in Mac OS X for iPython to behave [blahblah] (pandas_venv) % pip install ipython [blah blah] (pandas_venv) % pip install numpy [lots of…

The normal convention is to avoid using pylab, and instead use matplotlib directly.

Pylab is handy if you're just transitioning from Matlab, but otherwise, there's no reason to use it. It's a gigantic namespace, and all but a couple of functions are from numpy and matplotlib.pyplot.

Just do:

    import matplotlib.pyplot as plt
Instead of:

    import pylab as pl
Of course, in the end it's personal preference. As long as you don't need to know where things come from, then using pylab is fine.

Re: R and pandas and what I've learned about each

#17

Post by xradionut »

jmduke wrote: post #16

The creator of pandas wrote a book, Python on Data Analysis , which covers NumPy and Pandas. I found it an excellent primer. http://oreilly.com/shop/product/0636920023784.html

I'm enjoying working through this book. I admit I got it to fulfill a need, to quickly analyze huge text files before squeezing them into a RDMS and data warehouse for "proper" analysis and reporting. (Which is a more time consuming process require all sorts of meetings, approvals and effort.) A couple of fellow MS-DBAs threw a "shit-fit" when they saw it on my desk. (NIH Stockholm syndrome?)

Re: R and pandas and what I've learned about each

#18

Post by bsg75 »

Are there any comments as to the maturity of Pandas as compared to R?

I am used to the Python syntax, and while R is another language to learn, my assumption is that for data analysis its age compared to Pandas implies stability.

I could of course be wrong.

Re: R and pandas and what I've learned about each

#19

Post by takluyver »

teoruiz wrote: post #11

To install the whole set of Python modules needed and iPython in a virtualenv (trick: there is no "pylab" module to install): % virtualenv --distribute --no-site-packages pandas_venv [blahblah] % . pandas_venv/bin/activate (pandas_venv) % easy_install readline # Probably only needed in Mac OS X for iPython to behave [blahblah] (pandas_venv) % pip install ipython [blah blah] (pandas_venv) % pip install numpy [lots of…

Installing numpy with pip isn't recommended: it might not work (if you don't have the necessary development headers to compile it), the resulting numpy might be slower (if it hasn't managed to compile against properly optimised libraries) and compiling from source isn't a very quick way to install it.

For most users, the easiest way to get set up is a complete Python distribution, like Anaconda, EPD or Python(x,y). See the Scipy Stack installation page:

http://scipy.github.com/install.html

Re: R and pandas and what I've learned about each

#20

Post by takluyver »

bsg75 wrote: post #18

Are there any comments as to the maturity of Pandas as compared to R? I am used to the Python syntax, and while R is another language to learn, my assumption is that for data analysis its age compared to Pandas implies stability. I could of course be wrong.

I've not had any problems with pandas' stability. Where the age difference shows is in the availability of specific statistical methods. The Python package 'statsmodels' is working on that, but it's some way behind R.
Post reply on HN