Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A “frozen” dictionary for Python

lwn.net

Post reply on HN
Print view View original

141–150 of 172 posts

Re: A “frozen” dictionary for Python

#141

Post by morshu9001 »

krick wrote: post #126
morshu9001 wrote: post #106

Earlier quoted context omitted.

You can't really tell though. Maybe the dict is frozen but the values inside aren't. C++ tried to handle this with constness, but that has its own caveats that make some people argue against using it.

Indeed. So I don't really understand what this proposal tries to achieve. It even explicitly says that dict → frozendict will be O(n) shallow-copy, and the contention is only about O(n) part. So… yeah, I'm sure they are useful for some cases, but as Raymond has said — it doesn't seem to be especially useful, and I don't understand what people ITT are getting excited about.

Maybe treating Python like a systems language, so applying the same reasoning for const in C++ and Rust to it

Re: A “frozen” dictionary for Python

#142

Post by tekne »

vlovich123 wrote: post #137
pkulak wrote: post #112

Earlier quoted context omitted.

This is why I love how Rust approached this; almost by accident to make borrow checking work. Every reference is either mutable or not, and (with safe code), you can't use an immutable reference to get a mutable reference anywhere down the chain. So you can slowly construct a map through a mutable reference, but then return it out of a function as immutable, and that's the end of it. It's no longer ever mutable, and…

You cannot return an immutable version. You can return it owned (in which case you can assign/reassign it to a mut variable at any point) or you can take a mut reference and return an immutable reference - but whoever is the owner can almost always access it mutably.

I mean, if you return an immutable reference, the owner in fact cannot mutate it until that reference is dropped.

If you in fact return e.g. an Rc::new(thing) or Arc::new(thing), that's forever (though of course you can unwrap the last reference!)

Re: A “frozen” dictionary for Python

#143

Post by BiteCode_dev »

instig007 wrote: post #127
BiteCode_dev wrote: post #107

Earlier quoted context omitted.

pyrsistent is super slow, though. Just ran a quick benchmark: - Creation - 8-12x slower - Lookup - 22-27x slower - Contains check - 30-34x slower - Iteration - 5-14x slower - Merge - 32-158x slower Except at 10k+ items, batchup dates on 100K+ items or inserting 100 keys. This is rarely the case in practice, most dictionaries and dict operations are small, if you have a huge dict, you probably should be chunking your…

> pyrsistent is super slow, though Since when is Python about speed? > Just ran a quick benchmark Where's the code? Have you observed the bottleneck call? > Except at 10k+ items, batchup dates on 100K+ items or inserting 100 keys. > This is rarely the case in practice Where's the stats on the actual practice? > You'd better have an incredible ROI to justify that. The ROI being: fearless API design where 1) multiple i…

Clearly the ROI is perfect for you.

I won't waste more of your time.

Re: A “frozen” dictionary for Python

#144

Post by tylerhou »

cylemons wrote: post #82
perrygeo wrote: post #75

Python discovers immutable data, and gets it wrong. Frozendict is a blunt instrument - instead of a mutable free-for-all, it just locks the data for the entire lifecycle. Brace for the wave of code littered with deep copies, proudly proclaiming how functional it all is. If you want real immutable data structures, not a cheap imitation, check out pyrsistent.

How else would you "modify" immutable data other than by copying it?

Basically, a proxy. You don't need to deep copy; you just need to return a proxy object that falls back to the original dict if the key you are requesting has not been found or modified.

Functional data structures essentially create a proxy on every write. This can be inefficient if you make writes in batches, and you only need immutability between batches.

Re: A “frozen” dictionary for Python

#145

Post by instig007 »

BiteCode_dev wrote: post #143
instig007 wrote: post #127

Earlier quoted context omitted.

> pyrsistent is super slow, though Since when is Python about speed? > Just ran a quick benchmark Where's the code? Have you observed the bottleneck call? > Except at 10k+ items, batchup dates on 100K+ items or inserting 100 keys. > This is rarely the case in practice Where's the stats on the actual practice? > You'd better have an incredible ROI to justify that. The ROI being: fearless API design where 1) multiple i…

Clearly the ROI is perfect for you. I won't waste more of your time.

It's perfect for most Python developers actually, not just for myself, contrary to your "in practice" claim.

Re: A “frozen” dictionary for Python

#147

Post by FreakLegion »

zahlman wrote: post #39
pansa2 wrote: post #20

I wonder whether Raymond Hettinger has an opinion on this PEP. A long time ago, he wrote: "freezing dicts is a can of worms and not especially useful". https://mail.python.org/pipermail/python-dev/2006-February/0...

> Another PEP 351 world view is that tuples can serve as frozenlists; however, that view represents a Liskov violation (tuples don't support the same methods). This idea resurfaces and has be shot down again every few months. ... Well, yes; it doesn't support the methods for mutation . Thinking of ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance re…

> I've felt like frozendict was missing for a long time, though.

Type the dict as a mapping when you want immutability:

  x: Mapping[int, int] = {1: 1}

  x[1] = 2  # Unsupported target for indexed assignment ("Mapping[int, int]").
The only problem I've seen with this is:

  y = {}
  y[x] = 0  # Mypy thinks this is fine. Mapping is hashable, after all!
The issue here is less that dict isn't hashable than that Mapping is, though.

Re: A “frozen” dictionary for Python

#148

Post by emil-lp »

vlovich123 wrote: post #139

I’m curious why the dict->frozendict operation is not an O(1) operation when there’s a refcnt of 1 on the dict - that resolves the spooky action at a distance problem raised and solves the performance for the most common usage pattern (build up the dict progressively and convert to a frozendict for concurrent reads).

Because optimizations are not added to the PEP, but added later.

Re: A “frozen” dictionary for Python

#149

Post by emil-lp »

shadowgovt wrote: post #79

Regarding the spooky-action-at-a-distance concerns of a `.freeze()` method on dict: `.freeze()` should probably just return a frozendict instead of in-place mutating the dict, and they should be separate types. Under the hood, you'll have to build the hashtable anyway to make the frozendict; as long as you're doing that work, you may as well build an object to contain the hashtable and just have that object be separa…

The point is that freeze could work in constant time, whereas the copying takes linear time.

Another alternative mentioned was `move`, which would create a frozen version in constant time and clear the original dict.

Re: A “frozen” dictionary for Python

#150

Post by sam_bristow »

morshu9001 wrote: post #104
zahlman wrote: post #62

Earlier quoted context omitted.

> Ordering, like stability in sorting, is an incredibly useful property. I can't say I've noticed any good reasons to rely on it. Didn't reach for `OrderedDict` often back in the day either. I've had more use for actual sorting than for preserving the insertion order.

Same. Recently I saw interview feedback where someone complained that the candidate used OrderedDict instead of the built-in dict that is now ordered, but they'll let it slide... As if writing code that will silently do different things depending on minor Python version is a good idea.

Honestly, if I was writing some code that depended on dicts being ordered I think I'd still use OrderedDict in modern Python. I gives the reader more information that I'm doing something slightly unusual.
Post reply on HN