Live data from Hacker News

A Python dict that can report which keys you did not use

peterbe.com

41–45 of 45 posts

Re: A Python dict that can report which keys you did not use

#41
post #19

Earlier quoted context omitted.

Interesting! Can you elaborate a little bit more on your implementation?

Mine was a bit more specific. I had a JSON object of data exported per account I was importing, and then a complex mapping (also JSON) of where to put each piece of data. Therefore, I really wanted to know that I was actually pulling in all of the data I needed, so I tracked what was seen vs not seen, and compared against what was attempted to see. In the end it was basically a wrapper around the JSON object itself,…

Nice very interesting, thank you very much for taking the time to explain a bit further

Re: A Python dict that can report which keys you did not use

#43

Earlier quoted context omitted.

dict doesn't follow the usual object protocol, and overloaded methods are runtime dependent. It's only guaranteed that non-overloaded methods are resolved least surprisingly.

I think you mean overridden (i.e. defined in both base class and derived class) rather than overloaded (i.e. defined more than once in a single place but with different argument types, as least from a typing point of view [1]). Your comment seriously confused me till I figured that out. [1] https://typing.python.org/en/latest/spec/overload.html Even then, to be honest I'm a bit sceptical. Can you point at a link in t…

See Chapter 14 of "Fluent Python", 2nd Edition by Luciano Ramalho. He details this under the heading "Subclassing Built-In Types Is Tricky."

UserDict isn't just some historical artifact of a bygone era like some of the posters below are miscorrecing me on.

Re: A Python dict that can report which keys you did not use

#44
Does this handle nested dicts (in pickles in sql, which I had to write code to survey one time)?

A queue-based traversal has flatter memory utilization for deeply nested dicts than a recursive traversal in Python without TCO.

Given a visitor pattern traversal, a visit() function can receive the node path as a list of path components, and update a Counter() with a (full,path,tuple) or "delimiter\.escaped.path" key.

Python collections.UserDict implements the methods necessary to proxy the dict Mapping/MutableMapping interface to self.data. For dicts with many keys, it would probably be faster to hook methods that mutate the UserDict.data dict like __setitem__, get, setdefault, update() and maybe __init__() in order to track which keys have changed instead of copying keys() into a set to do an unordered difference with a list.

React requires setState() for all mutations this.state because there's no way to hook dunder methods in JS: setState() updates this.state and then notifies listeners or calls a list of functions to run when anything in this.state or when a value associated with certain keys or nested keys in this.state changes.

FWIU ipyflow exposes the subscriber refcount/reflist but RxPy specifically does not: ipyflow/core/test/test_refcount.py: https://github.com/ipyflow/ipyflow/blob/master/core/test/tes...

Anyways,

For test assertions, unittest.mock MagicMock can track call_count and call_args_list on methods that mutate a dict like __getitem__ and get(). There's also mock_calls, which keeps an ordered list of the args passed: https://docs.python.org/3/library/unittest.mock.html

Re: A Python dict that can report which keys you did not use

#45

Earlier quoted context omitted.

I think you mean overridden (i.e. defined in both base class and derived class) rather than overloaded (i.e. defined more than once in a single place but with different argument types, as least from a typing point of view [1]). Your comment seriously confused me till I figured that out. [1] https://typing.python.org/en/latest/spec/overload.html Even then, to be honest I'm a bit sceptical. Can you point at a link in t…

See Chapter 14 of "Fluent Python", 2nd Edition by Luciano Ramalho. He details this under the heading "Subclassing Built-In Types Is Tricky." UserDict isn't just some historical artifact of a bygone era like some of the posters below are miscorrecing me on.

It's unreasonable to ask me to chase down a reference in a book but I did it anyway because I was really curious.

It doesn't support what you said in your previous comment. It seemed to suggest that any calls to your overridden methods might magically use the base class instead, even when directly called from application code. But the book says something very different:

> The code of the built-ins (written in C) usually does not call methods overridden by user-defined classes.

But that is nothing to do with them being built-in classes or even being written in C. Any class, even written in pure Python, is not guaranteed to implement any of its methods in terms of other public methods, even when it's possible to do that. It just depends on how it's implemented.

Indeed that's true even for UserDict. It's no longer implemented in pure Python, but if you look at the 2.7 version [1], which was, you can see some methods implemented in terms of others (e.g. get() uses `key not in self` and `self[key]`) and others that aren't (e.g. keys() just uses self.dict.keys() rather than being implemented in terms of self.items()).

There was even a breaking change in Python 3.12 to which other methods UerDict.__getitem__() uses in its implementation [2]. I can sort of see some utility to UserDict but it seems at least as unpredictable as deriving from built in dict so it doesn't really buy you much. Either way, the only really safe way to use them is to override all methods that you want to behave differently.

[1] https://github.com/enthought/Python-2.7.3/blob/master/Lib/Us...

[2] https://github.com/python/cpython/issues/105524

Post reply on HN