Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python dicts are now ordered

softwaremaniacs.org

Post reply on HN
Print view View original

351–360 of 457 posts

Re: Python dicts are now ordered

#351

Post by to11mtm »

Someone wrote: post #117
rob74 wrote: post #84

Earlier quoted context omitted.

The Go designers went the other way (as they often do): > When iterating over a map with a range loop, the iteration order is not specified and is not guaranteed to be the same from one iteration to the next. Actually it is not only "not guaranteed to be the same", the runtime actively makes sure that the iteration order is actually different so you don't even start to rely on it...

That’s fairly common, as it is protection against denial-of-service attacks that use hash collisions to make code perform poorly ( https://events.ccc.de/2011/12/28/crypto-talk-at-28c3-effecti... ) If I read https://lwn.net/Articles/474912/ correctly, Perl fixed that in 2003. C#, Java and Swift do it, too. As does (or, reading this, did?) Python ( https://bugs.python.org/issue13703 )

> C#, Java and Swift do it, too.

Welllllllll

In the case of C# it is/was more a matter of picking the -correct- Dictionary.

'OrderedDictionary' has been around since Net 2.0 (2005 or so,) as has 'SortedDictionary'

But for some ungodly reason, there is no 'OrderedDictionary' included in the framework that you can use. All you get normally is the non-generic one (where you have to type-check everything...) There are internal implementations used by the framework, but you'd have to do some hacky things to even get it to work.

However, in my 12 years of C# programming, I've only run into the need for this _once_, and it was to assist interfacing with a VB6 monstrosity.

Re: Python dicts are now ordered

#352

Post by user5994461 »

d0mine wrote: post #318
squeaky-clean wrote: post #303

Earlier quoted context omitted.

There's also the indication that this is official, and not just an implementation quirk. I still used OrderedDict in 3.6 because there was no guarantee 3.7 (or later) wouldn't silently break ordering on the standard dict. At least now it's unlikely to change because it will be considered a breaking change.

BDFL declared Python dict to be ordered in 2017 — It can't be more official than that for Python https://mail.python.org/pipermail/python-dev/2017-December/1... i.e., Python 3.7 (whatever implementation must keep the insertion order). CPython keeps the order since Python 3.6. Pypy even before that.

A barely legible email dump with 50 concurrent answers is supposed to be the clear official statement?

Not to mention that developers just begin migrating to python 3 in 2017 and code still had to work on 2.7 that doesn't order.

Re: Python dicts are now ordered

#353

Post by true_religion »

user5994461 wrote: post #352
d0mine wrote: post #318

Earlier quoted context omitted.

BDFL declared Python dict to be ordered in 2017 — It can't be more official than that for Python https://mail.python.org/pipermail/python-dev/2017-December/1... i.e., Python 3.7 (whatever implementation must keep the insertion order). CPython keeps the order since Python 3.6. Pypy even before that.

A barely legible email dump with 50 concurrent answers is supposed to be the clear official statement? Not to mention that developers just begin migrating to python 3 in 2017 and code still had to work on 2.7 that doesn't order.

I knew about it, because everyone was talking about it. The origination of the actual information is irrelevant. It was basically interesting gossip for a few weeks.

Re: Python dicts are now ordered

#354

Post by philwelch »

jwkane wrote: post #326
philwelch wrote: post #296

Earlier quoted context omitted.

From what I recall of the JSON standard itself, there's no guarantee about key ordering being significant. If you're diffing serialized output to compare two JSON objects you need to be serializing it in a consistent format, otherwise even whitespace is going to throw you off.

It's significant to a human that wants to know what has changed.

If a human is inspecting serialized JSON using pen and paper, the human is presumably clever enough to match up key for key regardless of ordering.

If the human is using a computer to compare two JSON payloads (as the use of a diffing algorithm suggests), the human and computer should be clever enough as a team to realize that they could just deserialize and reserialize each JSON payload such that the keys were lexicographically sorted and the data was pretty-printed in the exact same way before running it through the diffing algorithm. `jq -Sc` would do the trick.

Re: Python dicts are now ordered

#355

Post by anderskaseorg »

user5994461 wrote: post #352
d0mine wrote: post #318

Earlier quoted context omitted.

BDFL declared Python dict to be ordered in 2017 — It can't be more official than that for Python https://mail.python.org/pipermail/python-dev/2017-December/1... i.e., Python 3.7 (whatever implementation must keep the insertion order). CPython keeps the order since Python 3.6. Pypy even before that.

A barely legible email dump with 50 concurrent answers is supposed to be the clear official statement? Not to mention that developers just begin migrating to python 3 in 2017 and code still had to work on 2.7 that doesn't order.

The official release notes for the following release is certainly a clear official statement.

https://www.python.org/downloads/release/python-370/

“The insertion-order preservation nature of dict objects is now an official part of the Python language spec.” (June 27, 2018)

Re: Python dicts are now ordered

#356

Post by viraptor »

yurlungur wrote: post #195
hprotagonist wrote: post #47

Earlier quoted context omitted.

If you know you're supporting old code, use OrderedDict. you arguably ought to anyway, for explicitness.

Yeah I think this is probably my main issue. I don't think it's reasonable to ask users of your code to always use 3.7+ instead of 3.6 if they are usually expected to be compatible. And it's also unnecessary to break such compatibility for something like preferring dict over OrderedDict anyways. At least I would try to avoid any such issues by still using OrderedDict. That said, I have no idea about the internals of…

Using OrderedDict is actually nice in this case, even if the default dict has the same ordering. That way you're explicitly saying you rely on that behaviour and it makes reading the code easier.

Re: Python dicts are now ordered

#357

Post by nicoburns »

tene wrote: post #298
skrebbel wrote: post #223

This is awesome, because the ordered map is the best data structure out there for easy & predictable programming, possibly only barring the array. There's so many cases where it's a benefit for map entries to retain order (and none where it's a problem). PHP really got this one right (and immediately messed it up by mixing ordered maps with arrays into a big soup, but hey, PHP). And so did, JS, sorta-kinda-by-acciden…

Could you explain some examples of what this is useful for? What sort of algorithms or operations do you have in mind where you both want insertion order, and also key-based lookup in the same data structure? I've made heavy use of all kinds of maps, and of queues and channels and arrays, but I don't recall ever noticing a situation where I wanted the properties of both mixed into the same data structure. I'd love to…

I wrote a program that read in a proprietary data format, ran a few transformations, then outputted CSVs. Using an ordered map was super-useful, because it gave me deterministic output between runs. Otherwise I ended up with CSVs with differently ordered rows...

Re: Python dicts are now ordered

#358

Post by [deleted] »

jwkane wrote: post #326
philwelch wrote: post #296

Earlier quoted context omitted.

From what I recall of the JSON standard itself, there's no guarantee about key ordering being significant. If you're diffing serialized output to compare two JSON objects you need to be serializing it in a consistent format, otherwise even whitespace is going to throw you off.

It's significant to a human that wants to know what has changed.

[deleted]

Re: Python dicts are now ordered

#359

Post by hencoappel »

masklinn wrote: post #112
hprotagonist wrote: post #47

Earlier quoted context omitted.

If you know you're supporting old code, use OrderedDict. you arguably ought to anyway, for explicitness.

OrderedDict is slow and expensive though: it maintains ordering through a doubly linked list. It has useful features for manipulating ordering but while I've regularly needed had use for maintaining insertion ordering I can't remember ever needing to move items around within a map.

If memory serves me correctly, ever since dicts became ordered the OrderedDict simply became a subclass of dict, so it will have exactly the same performance characteristics.

Re: Python dicts are now ordered

#360

Post by xapata »

ehsankia wrote: post #254
zurtex wrote: post #109

Earlier quoted context omitted.

No, OrderedDict has it's own C implementation which was created just before it was decided that dict would preserve order across iteration. Further there is a big difference, regular dict preserves order across iteration but OrderedDict treats order up to equality. I.e. this returns True: {1: 1, 2: 2} == {2: 2, 1: 1} Where as this returns False: OrderedDict({1: 1, 2: 2}) == OrderedDict({2: 2, 1: 1}) To make that diff…

For the former, I realize that's how it's done now, but there's nothing forcing it to stay that way. For the latter issue, as explained above, can't they just implement a replacement __eq__ only for OrderedDict, and still re-use the new dict implementation? Similarly, any subtle difference can be shimmed on top of the new implementation inside Python, no?

Depends what OrderedDict was being used for. If used for some of the obscure ordering features, replacement with dict + a shim might be a slowdown.
Post reply on HN