I've used PrettyTable on a few projects and found it to be very easy to use. Highly recommended!
Tabulate is also a good alternative, and more recent: https://pypi.python.org/pypi/tabulate
I would consider Tabulate much superior to PrettyTable.
Lesser-Known Python Data Analysis Libraries
31–40 of 69 posts
Re: Lesser-Known Python Data Analysis Libraries
#32My 2 cents: I would not recommend basing any new work on MRjob. As someone who inherited and has been maintaining a bunch of code that depends on it, the library seems to be barely maintained, support for VPC is only partial and not very well documented, the auditing tools stopped working quite a while ago and tracking the progress/status of EMR jobs is extremely painful (to be fair, this is more of an issue with Ela…
Do you know of any good alternatives? Any way to write MapReduces in python?
Re: Lesser-Known Python Data Analysis Libraries
#33dataset https://dataset.readthedocs.org/en/latest/ is pretty good and for most scenarios as easy as tinydb, but backed by a real SQL database.
Re: Lesser-Known Python Data Analysis Libraries
#34My 2 cents: I would not recommend basing any new work on MRjob. As someone who inherited and has been maintaining a bunch of code that depends on it, the library seems to be barely maintained, support for VPC is only partial and not very well documented, the auditing tools stopped working quite a while ago and tracking the progress/status of EMR jobs is extremely painful (to be fair, this is more of an issue with Ela…
Do you know of any good alternatives? Any way to write MapReduces in python?
Yes! Check out dask: http://www.slideshare.net/continuumio
Its free with a permissive license.
It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops.
DISCLAIMER: I don't work for continuum. I just want to see its projects succeed because I was a user will benefit.
Re: Lesser-Known Python Data Analysis Libraries
#35Earlier quoted context omitted.
Do you know of any good alternatives? Any way to write MapReduces in python?
Unfortunately, no. We are slowly moving away to a streaming infrastructure, so I've been mostly trying to "keep it running" until we are done replacing it. Sorry.
Check out dask: http://www.slideshare.net/continuumio
Its free with a permissive license and actively growing.
It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops.
DISCLAIMER: I don't work for continuum. I just want to see its projects succeed because I was a user will benefit.
Re: Lesser-Known Python Data Analysis Libraries
#36Check out dask for distributed and out of core parallel programming : http://www.slideshare.net/continuumio
Its free with a permissive license.
It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops.
DISCLAIMER: I don't work for continuum. I just want to see its projects succeed because I was a user will benefit.
Re: Lesser-Known Python Data Analysis Libraries
#37Earlier quoted context omitted.
Do you know of any good alternatives? Any way to write MapReduces in python?
Luigi does decent job. It is relatively easy to start with and powerful enough to do almost anything
I've been using Luigi for a few months, with no complaints. It supports running Python jobs on Hadoop and Spark, but it's not really a MapReduce framework unto itself.
However http://discoproject.org/ might be worth a look as a standalone alternative.
Re: Lesser-Known Python Data Analysis Libraries
#38Check out dask for distributed and out of core parallel programming : http://www.slideshare.net/continuumio Its free with a permissive license. It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops. DISCLAIMER: I don't work for continuum. I just want to see its pr…
I think you might be interested by this talk: https://www.youtube.com/watch?v=gVBLF0ohcrE
Re: Lesser-Known Python Data Analysis Libraries
#39My 2 cents: I would not recommend basing any new work on MRjob. As someone who inherited and has been maintaining a bunch of code that depends on it, the library seems to be barely maintained, support for VPC is only partial and not very well documented, the auditing tools stopped working quite a while ago and tracking the progress/status of EMR jobs is extremely painful (to be fair, this is more of an issue with Ela…
Do you know of any good alternatives? Any way to write MapReduces in python?
Andrew Montalenti did a great talk about scaling out Python at Parsely at the last PyData conference: https://www.youtube.com/watch?v=gVBLF0ohcrE
But TBH, after a certain scale you should really be asking whether or not you should be using Python.
Re: Lesser-Known Python Data Analysis Libraries
#40Earlier quoted context omitted.
Luigi does decent job. It is relatively easy to start with and powerful enough to do almost anything
I've been using Luigi for a few months, with no complaints. It supports running Python jobs on Hadoop and Spark, but it's not really a MapReduce framework unto itself. However http://discoproject.org/ might be worth a look as a standalone alternative.
I have used Disco extensively in the past, nothing but good things to say about it. Fast job launch, easy to write, the DFS has been stellar. This was only using Python for job code.