Live data from Hacker News

A “frozen” dictionary for Python

lwn.net

41–50 of 172 posts

Re: A “frozen” dictionary for Python

#41
post #30

Wow weird Mandela effect for me. I really remember this being a built and actually using it.

You may be thinking of the `frozenset()` built in or the third party Python module [frozendict](https://pypi.org/project/frozendict/)?

Personally, I’ve been using a wrapper around `collections.namedtuple` as an underlying data structure to create frozen dictionaries when I’ve needed something like that for a project.

Re: A “frozen” dictionary for Python

#42
post #6

A frozen dictionary would be very welcome. You can already do something similar using MappingProxyType [0] from types import MappingProxyType d = {} d["a"] = 1 d["b"] = 2 print(d) frozen = MappingProxyType(d) print(frozen["a"]) # Error: frozen["b"] = "new" [0]: https://docs.python.org/3/library/types.html#types.MappingPr...

> You can already do something similar

Only if you deny access to the underlying real dict.

Re: A “frozen” dictionary for Python

#43
post #17
post #7

If this gets wide enough use, they could add an optimization for code like this: n = 1000 a = {} for i in range(n): a[i] = str(i) a = frozendict(a) # O(n) operation can be turned to O(1) It is relatively easy for the JIT to detect the `frozendict` constructor, the `dict` input, and the single reference immediately overwritten. Not sure if this would ever be worth it.

Wouldn't ref-counting CPython already know that a has a single reference, allowing this optimization without needing any particular smart JIT?

I think GP was talking about optimizing away the O(N) call on the last line. The GC will take care of removing the reference to the old (mutable) dict, but constructing a new frozendict from a mutable dict would, in the current proposal, be an O(N) shallow copy.

There are also potentially other optimizations that could be applied (not specific to dict/frozendict) to reduce the memory overhead on operations like "a = f(a)" for selected values of "f".

Re: A “frozen” dictionary for Python

#44
Can someone give a strong rationale for a separate built-in class? Because "it prevents any unintended modifications" is a bit weak.

If you have fixed keys, a frozen dataclass will do. If you don't, you can always start with a normal dict d, then store tuple(sorted(d.items())) to have immutability and efficient lookups (binary search), then throw away d.

Re: A “frozen” dictionary for Python

#45
post #8

Great! Now make `set` have a stable order and we're done here.

Aren’t sets unsorted by definition? Or do repeated accesses without modification yield different results?

Not related to Python, but one of the possible implementations of a set, i.e. of an equivalence class on sequences, is as a sorted array (with duplicates eliminated, unless it is a multiset, where non-unique elements are allowed in the sorted array), as opposed to the unsorted array that can store an arbitrary sequence.

So sets can be viewed as implicitly sorted, which is why the order of the elements cannot be used to differentiate two sets.

Being sorted internally to enforce the equivalence between sets with elements provided in different orders does not imply anything about the existence of an operation that would retrieve elements in a desired order or which would return subsets less or greater than a threshold. When such operations are desired, an order relation must be externally defined on the set.

So a possible definition of sets and multisets is as sorted arrays with or without unicity of elements, while sequences are unsorted arrays (which may also have the constraint of unique elements). However the standard set operations do not provide external access to the internal order, which is an order between arbitrary identifiers attached to the elements of the set, which have no meaning externally.

Re: A “frozen” dictionary for Python

#46
This subject always seems to get bogged down in discussions about ordered vs. unordered keys, which to me seems totally irrelevant. No-one seems to mention the glaring shortcoming which is that, since dictionary keys are required to be hashable, Python has the bizarre situation where dicts cannot be dict keys, as in...

{{'foo': 'bar'}: 1, {3:4, 5:6}: 7}

...and there is no reasonable builtin way to get around this!

You may ask: "Why on earth would you ever want a dictionary with dictionaries for its keys?"

More generally, sometimes you have an array, and for whatever reason, it is convenient to use its members as keys. Sometimes, the array in question happens to be an array of dicts. Bang, suddenly it's impossible to use said array's elements as keys! I'm not sure what infuriates me more: said impossibility, or the python community's collective attitude that "that never happens or is needed, therefore no frozendict for you"

Re: A “frozen” dictionary for Python

#47
post #17
post #7

If this gets wide enough use, they could add an optimization for code like this: n = 1000 a = {} for i in range(n): a[i] = str(i) a = frozendict(a) # O(n) operation can be turned to O(1) It is relatively easy for the JIT to detect the `frozendict` constructor, the `dict` input, and the single reference immediately overwritten. Not sure if this would ever be worth it.

Wouldn't ref-counting CPython already know that a has a single reference, allowing this optimization without needing any particular smart JIT?

First thought: I would very much expect it to be able to do this optimization given the similar things it does for string concatenation.

But actually, I suspect it can't do this optimization simply because the name `frozendict` could be shadowed.

Re: A “frozen” dictionary for Python

#48

Earlier quoted context omitted.

Aren’t sets unsorted by definition? Or do repeated accesses without modification yield different results?

this is likely in reference to the fact that dicts have maintained insertion order since Python ~3.6 as property of the language. Mathematically there's no defined order to a set, and a dict is really just a set in disguise, but it's very convenient for determinism to "add" this invariant to the language.

Sets use a different implementation intentionally (i.e. they are not "a dict without values") exactly because it's expected that they have different use cases (e.g. union/intersection operations).

Re: A “frozen” dictionary for Python

#49
Concurrency is a good motivation, but this is super useful even in straight line code. There’s a huge difference between functions that might mutate a dictionary you pass in to them and functions that definitely won’t. Using Mapping is great, but it’s a shallow guarantee because you can violate it at run time.

Re: A “frozen” dictionary for Python

#50
post #39
post #20

I wonder whether Raymond Hettinger has an opinion on this PEP. A long time ago, he wrote: "freezing dicts is a can of worms and not especially useful". https://mail.python.org/pipermail/python-dev/2006-February/0...

> Another PEP 351 world view is that tuples can serve as frozenlists; however, that view represents a Liskov violation (tuples don't support the same methods). This idea resurfaces and has be shot down again every few months. ... Well, yes; it doesn't support the methods for mutation . Thinking of ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance re…

> ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance relationship.

Theoretically, could `set` be a subclass of `frozenset` (and `dict` of `frozendict`)? Do other languages take that approach?

> linking [immutability] more explicitly to hashability

AFAIK immutability and hashability are equivalent for the language's "core" types. Would it be possible to enforce that equivalence for user-defined types, given that mutability and the implementation of `__hash__` are entirely controlled by the programmer?

Post reply on HN