Modern Pandas (Part 2): Method Chaining

Post by ok_dad » Sun, May 01, 2022, 8:45 PM UTC

mjburgess wrote: post #22

zasdffaa wrote: post #21

Earlier quoted context omitted.

My python is rusty but IIRC it allows functional stuff (except for expression-only lambdas, boo). From https://docs.python.org/3/howto/functional.html it's got map/filter/currying and plenty more, what's misting in your view?

pattern matching expressions, syntax for partial application and composition, a typing system which can express structural types, a generalised list comprehension

    from functools import partial

    somefunc_arg1_arg2 = partial(somefunc, arg1, arg2)

    # mypy will typecheck this code and see `MyString()` has a `.read()` method, so it's a `Readable` even though it doesn't sub-class `Readable`

    from typing import Any, Protocol

    class Readable(Protocol):
        def read(self) -> Any: ...

    def read_something(something: Readable) -> Any:
        return something.read()

    class MyString:
        a_string: str = "something"
        def read(self) -> str:
            return self.a_string

    read_something(MyString())

Post by MichaelRazum » Sun, May 01, 2022, 8:55 PM UTC

    .pipe(fetch, 'water')

    .pipe(fell_down, 'jack')

    .pipe(broke, 'crown')

    .pipe(tumble_after, 'jill')

Post by swied » Sun, May 01, 2022, 9:25 PM UTC

MichaelRazum wrote: post #43

I don't see how this ( df.pipe(went_up, 'hill') .pipe(fetch, 'water') .pipe(fell_down, 'jack') .pipe(broke, 'crown') .pipe(tumble_after, 'jill') ) is much better then something like that df = went_up(df, 'hill') df = fetch(df, 'water') df = fell_down(df, 'jack') df = broke(df, 'jack') df = tumble_after(df, 'jill') Would really like to hear an opinion about that.

    # Per store revenue for high price items   
    df_agg = (
      df1
      .query("unit_price > 400")
      .groupby('store_id','store_name')
      .agg({'revenue':     np.sum, 
            'customer_id': 'nunique'})
    )

Post by hervature » Sun, May 01, 2022, 10:18 PM UTC

mayankkaizen wrote: post #27

Pandas is something that I wish I could avoid at any cost but I can't. There is simply no design philosophy. API is as ugly as it gets. I find it greatly unintuitive. It feels like a giant missmash of hacks on top of other hacks. Sometime I wish designers of Numpy or scikit-learn should have developed Pandas.

Post by oneoff786 » Sun, May 01, 2022, 10:45 PM UTC

swied wrote: post #45

MichaelRazum wrote: post #43

I don't see how this ( df.pipe(went_up, 'hill') .pipe(fetch, 'water') .pipe(fell_down, 'jack') .pipe(broke, 'crown') .pipe(tumble_after, 'jill') ) is much better then something like that df = went_up(df, 'hill') df = fetch(df, 'water') df = fell_down(df, 'jack') df = broke(df, 'jack') df = tumble_after(df, 'jill') Would really like to hear an opinion about that.

What about when the steps don't return a data frame object, like 'groupby'? Then, you will have to think about coming up a name other than 'df'. Save your mental energy for making comments. Here's an example... # Per store revenue for high price items df_agg = ( df1 .query("unit_price > 400") .groupby('store_id','store_name') .agg({'revenue': np.sum, 'customer_id': 'nunique'}) )