Live data from Hacker News

Python Type Hints – *args and **kwargs (2021)

adamj.eu

31–40 of 158 posts

Re: Python Type Hints – *args and **kwargs (2021)

#31

The ability of **kwargs to leave behind no proper documentation and silently swallow any invalid arguments has made us remove them entirely from our codebase. They're almost entirely redundant when you have dataclasses.

What about decorators, or wrappers around third-party code whose contracts change frequently (or even second party code when interacting with functions provided by teams that don't follow explicit argument typing guidelines, if you have that sort of culture)?

Usually the solutions range from a culture of “just don’t” to tests/mypy that have become increasingly stricter over the years, every time we’ve come a step further up the ladder. But I admit, it has taken quite some bridging to get there.

Moving to static Python in most places has dramatically improved the code and language.

Re: Python Type Hints – *args and **kwargs (2021)

#32

Earlier quoted context omitted.

My question is that can @warps warp more than 1 function? Maybe in some use case people need to merge 2 functions into 1, I don't know if it can handle this situation.

I'm not sure what it means to "merge two functions into one", can you elaborate? If you are referring to a type signature for a function that passes through it's arguments to one of two inner functions, each of which has different signatures, such that the outer signature accepts the union of the two inner signatures, well ... you could achieve that with ParamSpecs or similar, but it would be pretty hard to read and…

> I'm not sure what it means to "merge two functions into one", can you elaborate?

I'm not OP, but I see this pattern often enough:

    def foo(**kwargs):
        pass

    der bar(**kwargs):
        pass

    def wrapper(**kwargs):
        foo(**kwargs)
        bar(**kwargs)

Re: Python Type Hints – *args and **kwargs (2021)

#33
post #7

Earlier quoted context omitted.

It is used when the number of argument can vary, like: def sum(*args: int) -> int: if len(args) == 0: return 0 return args[0] + sum(*args[1:])

It seems altogether surprising that with an empty list or tuple a, a[1] results in index error, yet a[1:] quietly returns an empty list or tuple.

> It seems altogether surprising that with an empty list or tuple a, a[1] results in index error, yet a[1:] quietly returns an empty list or tuple.

`a[1:]` returns the sequence of elements that start at index 1. If there is no such element, the list is empty. I don’t see any good reason why this should throw an error.

Re: Python Type Hints – *args and **kwargs (2021)

#34
post #20

Although these two comes in handly, people have been using them wrong. Often in scientific open source package, they slap *kwargs in function definition without documentation. How am I suppose to know what to pass in? https://qiskit.org/ecosystem/aer/stubs/qiskit_aer.primitives...

Especially when they don't even leave a doc string so you're forced to track down the packages documentation online just to interact with certain interfaces. I work in a large python codebase, we have almost no usage of `*kwargs` beyond proxy methods because of the nature of how they obfuscate the real interface for other developers.

The worst is when someone puts **kwargs at the base of a class hierarchy, not only necessitating its use in subclasses (if you want to be strict about types) but also swallowing errors for no good reason. Fortunately I think this style is fading out as type hints become more popular.

Re: Python Type Hints – *args and **kwargs (2021)

#35
post #29
post #20

Earlier quoted context omitted.

Especially when they don't even leave a doc string so you're forced to track down the packages documentation online just to interact with certain interfaces. I work in a large python codebase, we have almost no usage of `*kwargs` beyond proxy methods because of the nature of how they obfuscate the real interface for other developers.

When I was first starting out, a then senior engineer told me: "friends don't let other friends use kwargs". That always stuck with me.

I once worked on a code base where we had *kwargs passed down 4 or 5 layers deep (not my idea.) It was a true joy.

Re: Python Type Hints – *args and **kwargs (2021)

#36
> In the function body, args will be a tuple, and kwargs a dict with string keys.

This always bugs me: why is `args` immutable (tuple) but `kwargs` mutable (dict)? In my experience it’s much more common to have to extend or modify `kwargs` rather than `args`, but I would find more natural having an immutable dict for `kwargs`.

Re: Python Type Hints – *args and **kwargs (2021)

#37
post #16

Although these two comes in handly, people have been using them wrong. Often in scientific open source package, they slap *kwargs in function definition without documentation. How am I suppose to know what to pass in? https://qiskit.org/ecosystem/aer/stubs/qiskit_aer.primitives...

Sadly a problem with any wrapper function is that it nullifies this kind of information. Use functools.wraps.

PyCharm usually figured this out if it's not too complex. I often wrap session.request() with some defaults/overrides and autocomplete usually shows me the base arguments as well.

Re: Python Type Hints – *args and **kwargs (2021)

#38
post #25

The ability of **kwargs to leave behind no proper documentation and silently swallow any invalid arguments has made us remove them entirely from our codebase. They're almost entirely redundant when you have dataclasses.

What do you do when inheriting from a base class with a defined __init__ ?

For everybody reading this and scratching their head why this is relevant: Python subclassing is strange.

Essentially super().__init__() will resolve to a statically unknowable class at run-time because super() refers to the next class in the MRO. Knowing what class you will call is essentially unknowable as soon as you accept that either your provider class hierarchy may change or you have consumers you do not control. And probably even worse, you aren't even guaranteed that the class calling your constructor will be one of your subclasses.

Which is why for example super().__init__() is pretty much mandatory to have as soon as you expect that your class will be inherited from. That applies even if your class inherits only from object, which has an __init__() that is guaranteed to be a nop. Because you may not even be calling object.__init__() but rather some sibling.

So the easiest way to solve this is: Declare everything you need as keyword argument, but then only give **kwargs in your function signature to allow your __init__() to handle any set of arguments your children or siblings may throw at you. Then remove all of "your" arguments via kwargs.pop('argname') before calling super().__init__() in case your parent or uncle does not use this kwargs trick and would complain about unknown arguments. Only then pass on the cleaned kwargs to your MRO foster parent.

So while using **kwargs seems kind of lazy, there is good arguments, why you cannot completely avoid it in all codebases without major rework to pre-existing class hierarchies.

For the obvious question "Why on earth?" These semantics allow us to resolve diamond dependencies without forcing the user to use interfaces or traits or throwing runtime errors as soon as something does not resolve cleanly (which would all not fit well into the Python typing philosophy.)

Re: Python Type Hints – *args and **kwargs (2021)

#39
post #29

Earlier quoted context omitted.

When I was first starting out, a then senior engineer told me: "friends don't let other friends use kwargs". That always stuck with me.

I once worked on a code base where we had *kwargs passed down 4 or 5 layers deep (not my idea.) It was a true joy.

This is literally me. It is a math program that can evaluate equations and generate code. 6 layers of heterogeneous data structure which the math operation being act on 1st layer has its effect down to 6th layer. Temporarily using *kwargs to make it works but still thinking what is the proper way to do it right.

Re: Python Type Hints – *args and **kwargs (2021)

#40

Earlier quoted context omitted.

I'm not sure what it means to "merge two functions into one", can you elaborate? If you are referring to a type signature for a function that passes through it's arguments to one of two inner functions, each of which has different signatures, such that the outer signature accepts the union of the two inner signatures, well ... you could achieve that with ParamSpecs or similar, but it would be pretty hard to read and…

> I'm not sure what it means to "merge two functions into one", can you elaborate? I'm not OP, but I see this pattern often enough: def foo(**kwargs): pass der bar(**kwargs): pass def wrapper(**kwargs): foo(**kwargs) bar(**kwargs)

Yup, this exactly.
Post reply on HN