Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lesser-Known Python Data Analysis Libraries

jyotiska.github.io

Post reply on HN
Print view View original

61–69 of 69 posts

Re: Lesser-Known Python Data Analysis Libraries

#61

Post by tanlermin »

gshulegaard wrote: post #38
tanlermin wrote: post #36

Check out dask for distributed and out of core parallel programming : http://www.slideshare.net/continuumio Its free with a permissive license. It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops. DISCLAIMER: I don't work for continuum. I just want to see its pr…

I think you might be interested by this talk: https://www.youtube.com/watch?v=gVBLF0ohcrE

Thanks...though the whole "GIL being a feature" is sort of a joke.

Re: Lesser-Known Python Data Analysis Libraries

#62

Post by merlincorey »

Semiapies wrote: post #53
jbrambleDC wrote: post #49

Vincent has not been properly maintained in a year. and is broken at this point since the release of Vega 2.0

Yes, this recommendation puzzled me. It's essentially a dead project. "Vincent is essetially frozen for development right now, and has been for quite a while. The features for the currently targeted version of Vega (1.4) work fine, but it will not work with Vega 2.x releases. Regarding a rewrite, I'm honestly not sure if it's worth the time and effort at this point."

The new project is here: https://github.com/uwdata/ipython-vega-lite

Re: Lesser-Known Python Data Analysis Libraries

#64

Post by Bjartr »

philovivero wrote: post #6
melor wrote: post #2

I use histogram.py from https://github.com/bitly/data_hacks all the time...

Interesting. Looks similar to my version, albeit with a bit fewer features. https://github.com/philovivero/distribution

I expect if you mentioned a few of the features yours has that the other doesn't you wouldn't have gotten downvoted.

Re: Lesser-Known Python Data Analysis Libraries

#65

Post by pwang »

pvnick wrote: post #25
_dark_matter_ wrote: post #16

Earlier quoted context omitted.

https://hadoop.apache.org/docs/r1.2.1/streaming.html

This is likely the best answer for those who wish to code within the map/reduce paradigm by hand and would prefer to use python.

BUT WHY

Your performance is going to be complete and utter crap because you're paying for serialization on every single data element.

Dask is higher performance and more pythonic: http://matthewrocklin.com/blog/work/2016/02/22/dask-distribu...

Re: Lesser-Known Python Data Analysis Libraries

#66

Post by dguaraglia »

gdulli wrote: post #52
dguaraglia wrote: post #13

My 2 cents: I would not recommend basing any new work on MRjob. As someone who inherited and has been maintaining a bunch of code that depends on it, the library seems to be barely maintained, support for VPC is only partial and not very well documented, the auditing tools stopped working quite a while ago and tracking the progress/status of EMR jobs is extremely painful (to be fair, this is more of an issue with Ela…

I have the opposite experience with MrJob. Classifying it as an inactive project is demonstrably false. The rest are EMR complaints, I use it on my own Hadoop cluster.

Just read the comment from one of the creators: https://news.ycombinator.com/item?id=11528776

Re: Lesser-Known Python Data Analysis Libraries

#67

Post by gshulegaard »

tanlermin wrote: post #61
gshulegaard wrote: post #38

Earlier quoted context omitted.

I think you might be interested by this talk: https://www.youtube.com/watch?v=gVBLF0ohcrE

Thanks...though the whole "GIL being a feature" is sort of a joke.

I believe he meant it as a joke. At least that's how I interpreted it :)

Re: Lesser-Known Python Data Analysis Libraries

#68

Post by metaobject »

jventura wrote: post #30
metaobject wrote: post #10

I've used PrettyTable on a few projects and found it to be very easy to use. Highly recommended!

Tabulate is also a good alternative, and more recent: https://pypi.python.org/pypi/tabulate

Looks good! It might be time to update some code ...

Re: Lesser-Known Python Data Analysis Libraries

#69

Post by BrandoElFollito »

googletron wrote: post #54
c17r wrote: post #12

I'll have to check delorean out, I usually use http://crsmithdev.com/arrow/ for python date manipulation. It works a lot like the javascript library moment.

cool, let me know if you have any issues. sanx.

I use arrow for all my time related operations. I tried dolorean once (very quickly) and found out it was missing several elelents I needed (which arrow had). Maybe I did not look closely enough, I will try again and be back if there is interest. Thanks.
Post reply on HN