Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern Pandas (Part 2): Method Chaining

tomaugspurger.github.io

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Modern Pandas (Part 2): Method Chaining

#61

Post by [deleted] »

brundolf wrote: post #55
hervature wrote: post #47

Earlier quoted context omitted.

I always see these type of complaints and, when I actually sit down with people to resolve their aversion, it ultimately comes back to they use Pandas incorrectly or simply are not able to grok documentation. The usual dead giveaway is "Pandas documentation is horrible". Anyone who has used more than one documentation would know that documentation rarely includes every function in the API let alone the argument, exam…

In my relatively brief experience maintaining a Python web service backed by Pandas: It's basically a DSL constructed out of the dismembered syntactic bones of Python, which breaks every piece of semantics in the host language that it possibly can. I'm sure this is convenient (and maybe even tractable to use) in an interactive notebook, where you can try out and verify behavior in real-time by looking at the output.…

[deleted]

Re: Modern Pandas (Part 2): Method Chaining

#62

Post by jeffwass »

electroly wrote: post #59
jeffwass wrote: post #44

Earlier quoted context omitted.

A former coworker of mine was a huge fan of functional programming, and also deeply allergic to mutation. So if you reused variables like that you’d get an angry earful. Though if you replaced each subsequent line with df1 and df2 and so on he wouldn’t mind as much. I can’t opine as to whether one approach or the other is intrinsically better. But echoes of his tirades still ring when I see the same variable name red…

There's no mutation there; it's just rebinding the name. Rebinding is very different from mutation. It wouldn't be my stylistic choice either but your FP friend wouldn't be complaining about mutation here.

Thanks for clarifying, though he definitely used the word “mutation” in this type of rebinding scenario.

Re: Modern Pandas (Part 2): Method Chaining

#63

Post by mjburgess »

ok_dad wrote: post #41
mjburgess wrote: post #22

Earlier quoted context omitted.

pattern matching expressions, syntax for partial application and composition, a typing system which can express structural types, a generalised list comprehension

> pattern matching expressions Available in 3.10: https://peps.python.org/pep-0636/ > syntax for partial application... from functools import partial somefunc_arg1_arg2 = partial(somefunc, arg1, arg2) > ...and composition A native compositional syntax would be nice. > a typing system which can express structural types # mypy will typecheck this code and see `MyString()` has a `.read()` method, so it's a `Readable` ev…

they arent expressions, defeating the whole point

that isnt syntax for partial appication, and the protocol system for structrural typing is syntactically and practically absurd

the question is "why do data processing libs in python look syntactically illegible" and the answer is largely, as youve posted above, that python doesnt support useful syntax

Re: Modern Pandas (Part 2): Method Chaining

#64

Post by mjburgess »

biellls wrote: post #32
mjburgess wrote: post #22

Earlier quoted context omitted.

pattern matching expressions, syntax for partial application and composition, a typing system which can express structural types, a generalised list comprehension

Pattern matching is supported in 3.10 and can match classes structurally, as well as other types (edit: I see that you mean you don't like that it's a statement, which I agree with). The typing system supports structural types with Protocol. Personally what I miss most are multi-line lambdas.

I'm not asking for "support" i'm asking for syntax. The question is why the syntax of data processing in python is illegible, not "are things technically possible in python".

Commenters here are keen to say "but dont you know!" -- and, yes, I do.

Re: Modern Pandas (Part 2): Method Chaining

#65

Post by zasdffaa »

mjburgess wrote: post #64
biellls wrote: post #32

Earlier quoted context omitted.

Pattern matching is supported in 3.10 and can match classes structurally, as well as other types (edit: I see that you mean you don't like that it's a statement, which I agree with). The typing system supports structural types with Protocol. Personally what I miss most are multi-line lambdas.

I'm not asking for "support" i'm asking for syntax . The question is why the syntax of data processing in python is illegible, not "are things technically possible in python". Commenters here are keen to say "but dont you know!" -- and, yes , I do.

To be fair the confusion of FP vs FP syntax is from your comment: "Perhaps if python would add more support for FP (rather than its present hostility)"

Python does FP fine and FP via an ugly library or via elegant inbuilt syntax is still FP. I get you want nicer syntax but it wasn't clear.

I dunno, write a PEP and see if it gets support. I hope you succeed!

Re: Modern Pandas (Part 2): Method Chaining

#66

Post by tpoacher »

mayankkaizen wrote: post #27

Pandas is something that I wish I could avoid at any cost but I can't. There is simply no design philosophy. API is as ugly as it gets. I find it greatly unintuitive. It feels like a giant missmash of hacks on top of other hacks. Sometime I wish designers of Numpy or scikit-learn should have developed Pandas.

They have. It's called numpy records, and they're delightful.

The problem is nobody is using them.

Re: Modern Pandas (Part 2): Method Chaining

#68

Post by sterlinm »

lysecret wrote: post #42

Ok it seems I am the only person that loves pandas. And I have just recently started to like method chaining (even written some code to enable it with beautifulsoup) so this post came just at the right time.

Given how much of a role pandas seems to have played in the growth of Python over the last decade I suspect you aren't really the only person that loves pandas :)

I think you'll find a similar selection bias if you ask HN commenters what they think about Excel.

Re: Modern Pandas (Part 2): Method Chaining

#69

Post by sterlinm »

eesmith wrote: post #35
sterlinm wrote: post #26

This is a great series of articles but a bit funny to call it modern pandas these days since it’s six years old. Has it been updated?

Modern art ended in the 1970's. https://en.wikipedia.org/wiki/Modern_art Perhaps the successor should be contemporary Pandas, or postmodern Pandas. :)

Postmodern Pandas sounds like a great title for a blog post.

Re: Modern Pandas (Part 2): Method Chaining

#70

Post by lysecret »

sterlinm wrote: post #68
lysecret wrote: post #42

Ok it seems I am the only person that loves pandas. And I have just recently started to like method chaining (even written some code to enable it with beautifulsoup) so this post came just at the right time.

Given how much of a role pandas seems to have played in the growth of Python over the last decade I suspect you aren't really the only person that loves pandas :) I think you'll find a similar selection bias if you ask HN commenters what they think about Excel.

Good point, also there are a lot of similarities between Excel and Pandas as well.

I think this also is a fundamental distinction between normal SWEs and People who use Excel as well as Data Engineers.

You always start with data, and you have no control over it. So this means:

1. you need state base programming env (excel, Jupyter) 2. you need to look at it to see whats there (plots)

I guess HN is mostly comprised of SWEs who built the DBs and Websites that create the data that then gets consumed by the Data Engs and the Excel people :D

Post reply on HN