I've always found pandas really hard to use or reason about. I eventually get there but I don't like the code. Obviously subjective. I've never used another "data science" language though so I've no experience beyond it.
Modern Pandas (Part 2): Method Chaining
11–20 of 72 posts
Re: Modern Pandas (Part 2): Method Chaining
#12I've always found pandas really hard to use or reason about. I eventually get there but I don't like the code. Obviously subjective. I've never used another "data science" language though so I've no experience beyond it.
Everyone does. Perhaps if python would add more support for FP (rather than its present hostility), we'd be able to phrase data transformations more naturally in the language.
Re: Modern Pandas (Part 2): Method Chaining
#13Pandas may be OK for people who need to do a little data processing in their Python project, but I still recommend R and the tidyverse to those who need a serious tool of thought for analytics. Everything is so much more tidy and concise and intuitive and flexible. You can usually write code that directly expresses your high-level intent with a minimum of syntactic cruft. Full disclosure - the downside of modern R is…
There are a number of good packages in Python specializing in variations of powerful chained processing in Pandas. My own is this one: https://github.com/WinVector/data_algebra .
I like the general direction of this.
I do want to note that "there are many things for X in language Y" isn't necessarily a positive thing. It often means the community lacks the clarity of thinking or will to converge on one excellent product. Instead there are lots of okayish things each developed by a single person or handful of people.
Re: Modern Pandas (Part 2): Method Chaining
#14I've always found pandas really hard to use or reason about. I eventually get there but I don't like the code. Obviously subjective. I've never used another "data science" language though so I've no experience beyond it.
I'd recommend checking out polars as an alternative to pandas - https://github.com/pola-rs/polars
It has a rather different api, and is significantly faster. Highly recommend it.
Re: Modern Pandas (Part 2): Method Chaining
#15Love it. This is an excellent resource for brushing up on certain parts of the API I don't use as frequently. I've been using Pandas daily for about 6 years now and to see how it's evolved over time makes me really proud of the community. The obligatory comparisons to other tools in R are cliched and seem to completely miss the point.
Re: Modern Pandas (Part 2): Method Chaining
#16What always bothers me is you can't method chain an in-place apply of some function on a (subset of) columns in an elegant way. Pipe and assign make it possible, but definitely not nice, just look at the example code
I think I once proposed an apply_to method for this purpose but that got -1'nd by the creators in no time
Re: Modern Pandas (Part 2): Method Chaining
#17I've always found pandas really hard to use or reason about. I eventually get there but I don't like the code. Obviously subjective. I've never used another "data science" language though so I've no experience beyond it.
Pandas makes it really hard to do easy things, and really easy to do hard things. But its more useful to the people who need it to do the hard things.
Re: Modern Pandas (Part 2): Method Chaining
#18Pandas may be OK for people who need to do a little data processing in their Python project, but I still recommend R and the tidyverse to those who need a serious tool of thought for analytics. Everything is so much more tidy and concise and intuitive and flexible. You can usually write code that directly expresses your high-level intent with a minimum of syntactic cruft. Full disclosure - the downside of modern R is…
It's intuitive to you but I have always found R a real pain to deal with. I acknowledge all of Pandas' flaws but R is not for me, and I've used it a lot in the past. It's all subjective and dependent on the context and requirements of a project.
R is a mixed bag. Base R is like PHP - an organically grown mishmash of stale, ugly, inconsistent conventions, beloved only by people who have been trained in it or used it for decades. But the tidyverse has an extremely thoughtful API that is easier to learn and use than anything else in the data science world.
Re: Modern Pandas (Part 2): Method Chaining
#19I've always found pandas really hard to use or reason about. I eventually get there but I don't like the code. Obviously subjective. I've never used another "data science" language though so I've no experience beyond it.
I'd recommend checking out polars as an alternative to pandas - https://github.com/pola-rs/polars It has a rather different api, and is significantly faster. Highly recommend it.
The issue is there's a huge ecosystem developed on top of Pandas. So if it's different API, now you lose access to all the ecosystem.
Re: Modern Pandas (Part 2): Method Chaining
#20Quick pdb trick for pipe-ers, you can stick this in the middle:
.pipe(lambda df: (df, pdb.set_trace())[0])