It would be great to have **kwargs as frozendict by default. It could help with caching decorators to speedup key construction. For example natural key is simply `(args, kwargs)`.
A “frozen” dictionary for Python
121–130 of 172 posts
Re: A “frozen” dictionary for Python
#122Earlier quoted context omitted.
How else would you "modify" immutable data other than by copying it?
And, yet, somehow, functional languages have been doing this since forever ? I jest, I jest! There's an entire field of study focused on this! I'm no expert, but imagine you've got a singly-linked list, and we're going to allow only push_back() and pull_back() (no insert()). Let's go ahead and let multiple owners have at this list. If A does "pull_back()" what really happens is A has a local "tail" pointer that moves…
Interesting, I didn't think about that, so it is copying on write but on a more granular level
Re: A “frozen” dictionary for Python
#123Earlier quoted context omitted.
Ordering, like stability in sorting, is an incredibly useful property. If it costs a little, then so be it. This is optimizing for the common case, where memory is generally plentiful and dicts grow more than they shrink. Python has so many memory inefficiencies that occasional tombstones in the dict internal structure is unlikely to be a major effect. If you're really concerned, do `d = dict(d)` after aggressive del…
> Ordering, like stability in sorting, is an incredibly useful property. I can't say I've noticed any good reasons to rely on it. Didn't reach for `OrderedDict` often back in the day either. I've had more use for actual sorting than for preserving the insertion order.
It seems like opinions really differ on this item then. I love insertion sort ordering in mappings, and python with it was a big revelation. The main reason is that keys need some order, and insertion order -> iteration order is a lot better than pseudorandom order (hash based orders).
For me, it creates more reproducible programs and scripts, even simple ones.
Re: A “frozen” dictionary for Python
#124Earlier quoted context omitted.
Problem being that sets aren't consistently ordered and conversion to a tuple can result in an exponential (specifically, factorial) explosion in the number of possible keys associated with a single set. Nor can you sort all objects. Safe conversion of sets to tuples for use as keys is possible but the only technique I know requires an auxiliary store of objects (mapping objects to the order in which they were first…
tuple(sorted(s)) and if you can't even sort the values, they're probably not hashable. I get that this involves a copy, but so does frozenset, and you can cross that bridge in various ways if it's ever a problem.
Here are some types that support hashing:
str
bytes
int, float
complex
tuple
frozenset
I have crossed that bridge, and I'm telling you (again) that a sorted tuple is not a generic solution.
Re: A “frozen” dictionary for Python
#125Earlier quoted context omitted.
> consider an expression like '"pop" in dir(mySet)' class frozenset: pass class set(frozenset): def pop(self, key): pass I don't see why hasattr(mySet, 'pop') should be a problem here?
> I don't see why hasattr(mySet, 'pop') should be a problem here? I never said it's a problem (and I never said it's not!). I was specifically addressing two things: - The "theoretical" nature of the question I quoted (i.e. ignoring other aspects like subjectivity, practicality, convention, etc.) - The reasoning about "Liskov violation", which was quoted further up this thread. For context, here's Liskov's definition…
> I became rather jaded on the Liskov substitution principle after reading https://okmij.org/ftp/Computation/Subtyping
The root of the issue here is that Liskov substitution principle simply references ϕ(x) to be some property satisfied by objects of a class. It does not distinguish between properties that are designed by the author of the class to be satisfied or properties that happen to be satisfied in this particular implementation. But the Hyrum’s Law also states that properties that are accidentally true can become relied upon and as time passes become an intrinsic property. This to me suggests that the crux of the problem is that people don’t communicate sufficiently about invariants and non-invariants of their code.
Re: A “frozen” dictionary for Python
#126Earlier quoted context omitted.
It's interesting that he concludes that freezing dicts is "not especially useful" after addressing only a single motivation: the use of a dictionary as a key. He doesn't address the reason that most of us in 2025 immediately think of, which is that it's easier to reason about code if you know that certain values can't change after they're created. What a change in culture over the last 20 years!
You can't really tell though. Maybe the dict is frozen but the values inside aren't. C++ tried to handle this with constness, but that has its own caveats that make some people argue against using it.
Indeed. So I don't really understand what this proposal tries to achieve. It even explicitly says that dict → frozendict will be O(n) shallow-copy, and the contention is only about O(n) part. So… yeah, I'm sure they are useful for some cases, but as Raymond has said — it doesn't seem to be especially useful, and I don't understand what people ITT are getting excited about.
Re: A “frozen” dictionary for Python
#127Earlier quoted context omitted.
You're absolutely right: an "indirection data structure" is necessary. Freezing the data is the least interesting part - it doesn't give you any of the benefits typically associated with immutable data structures in functional languages. That's my point - Python is shipping a half solution that's being mistaken for a proper one. You think Python developers are going to roll their own HAMT on top of frozendicts? Or ar…
pyrsistent is super slow, though. Just ran a quick benchmark: - Creation - 8-12x slower - Lookup - 22-27x slower - Contains check - 30-34x slower - Iteration - 5-14x slower - Merge - 32-158x slower Except at 10k+ items, batchup dates on 100K+ items or inserting 100 keys. This is rarely the case in practice, most dictionaries and dict operations are small, if you have a huge dict, you probably should be chunking your…
> pyrsistent is super slow, though
Since when is Python about speed?
> Just ran a quick benchmark
Where's the code? Have you observed the bottleneck call?
> Except at 10k+ items, batchup dates on 100K+ items or inserting 100 keys.
> This is rarely the case in practice
Where's the stats on the actual practice?
> You'd better have an incredible ROI to justify that.
The ROI being: fearless API design where 1) multiple instances of high level components are truly independent and could easily parallelize, 2) calling sites know that they keep the original data intact and that callees behave within the immutability constraints, 3) default func inputs and global scope objects are immutable without having to implement another PEP, 4) collections are hashable in general.
Re: A “frozen” dictionary for Python
#128If this gets wide enough use, they could add an optimization for code like this: n = 1000 a = {} for i in range(n): a[i] = str(i) a = frozendict(a) # O(n) operation can be turned to O(1) It is relatively easy for the JIT to detect the `frozendict` constructor, the `dict` input, and the single reference immediately overwritten. Not sure if this would ever be worth it.
Wouldn't ref-counting CPython already know that a has a single reference, allowing this optimization without needing any particular smart JIT?
Indeed, the Tcl implementation does this so e.g. `set d [dict] ; dict set d key value` can modify d in place instead of creating a copy (since everything is immutable).
Re: A “frozen” dictionary for Python
#129I wonder whether Raymond Hettinger has an opinion on this PEP. A long time ago, he wrote: "freezing dicts is a can of worms and not especially useful". https://mail.python.org/pipermail/python-dev/2006-February/0...
> Another PEP 351 world view is that tuples can serve as frozenlists; however, that view represents a Liskov violation (tuples don't support the same methods). This idea resurfaces and has be shot down again every few months. ... Well, yes; it doesn't support the methods for mutation . Thinking of ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance re…
ImmutableFoo can't be a subclass of Foo, since it loses the mutator methods. But nor can Foo be a subclass of ImmutableFoo, since it loses the axiom of immutability (e.g. thread-safety) that ImmutableFoo has.
When you interpret Liskov substitution properly, it's very rare that anything Liskov-substitutes anything, making the entire property meaningless. So just do things based on what works best in the real world and aim for as much Liskov-substitution as is reasonable. Python is duck-typed anyway.
It's a decent guiding principle - Set and ImmutableSet are more substitutable than Set and Map, so Set deriving from ImmutableSet makes more sense than Set deriving from Map. It's just not something you can ever actually achieve.
Re: A “frozen” dictionary for Python
#130Earlier quoted context omitted.
tuple(sorted(s)) and if you can't even sort the values, they're probably not hashable. I get that this involves a copy, but so does frozenset, and you can cross that bridge in various ways if it's ever a problem.
Here are some types that support hashing: str bytes int, float complex tuple frozenset Aside from int and float, you cannot perform comparisons between objects of different types. Moreover, you cannot sort complex numbers at all. I have crossed that bridge, and I'm telling you (again) that a sorted tuple is not a generic solution.
I'm not saying the problem with tuple doesn't exist, but that there doesn't need to be a built-in way to deal with it. If for some unfortunate reason you've got a mixed-type set that you also want to use as a dict key, you can write a helper.