Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python dicts are now ordered

softwaremaniacs.org

Post reply on HN
Print view View original

291–300 of 457 posts

Re: Python dicts are now ordered

#291

Post by masklinn »

svnpenn wrote: post #210

It seems people dont realize that you can have your cake and eat it too. You can have both orderered and unordered maps in the same language: https://yaml.org/type/map https://yaml.org/type/omap No reason to argue about which one to make "dict". In fact it would be better to have both because youre taking a performance hit (a significant one) by ordering the entries.

Python has had both ordered and unordered maps in the same languages for 10 years or so (OrderedDict was added to the stdlib in 2009).

In 2016, the standard dict was moved to a new and more efficient implementation (more compact & faster to iterate). It was naturally ordered, and so as a side-effect of this change and since the CPython devs did not explicitly scramble it, iteration on dicts became both deterministic and specific (it would always follow insertion order).

This was considered both a boon because conserving iteration order is strictly more useful than not doing it and a woe because it would cause compatibility issues with third-party implementations or further changes down the line as users would absolutely start relying on the iteration order even if it was still considered unspecified.

So in the next major release they decided to resolve the issue by making this behaviour part of the language. It restricts future design space, but they can always add an unordered dict as part of the stdlib if that ever truly becomes useful so meh.

Re: Python dicts are now ordered

#292

Post by skrebbel »

philwelch wrote: post #286
skrebbel wrote: post #272

Earlier quoted context omitted.

Good for Ruby! Nice to hear. I'm pretty sure they didn't in Ruby ~1.8 though. It's been awhile :-) Can't remember exactly which version, but it bit me more than once so I'm pretty sure :)

Yup, I think Ruby 1.9 or so made hashes ordered.

Awesome! Thanks for the heads-up both of you :-)

Re: Python dicts are now ordered

#293

Post by taeric »

masklinn wrote: post #127
Sean1708 wrote: post #48

Earlier quoted context omitted.

It's extra annoying because collections.OrderedDict has been in the standard library for ages.

I fail to see what's annoying how. OrderedDict provide features for manipulating ordering, but they are expensive and slower. dict initially became ordered as a consequence of implementation details changes, and that was considered useful enough (or likely enough to trigger compatibility issues with third party implementation) that the core team decided to standardise it rather than break it (one of the advantages of…

Annoying is that it is not obvious why the two exist, by their name. They are both ordered. One is orderable in that you can change is order. I would not guess that looking at this.

For a language that prides itself on being readable, that is an amusing quirk.

Re: Python dicts are now ordered

#294

Post by taeric »

duckerude wrote: post #115
taeric wrote: post #10

This is a solid argument for why you should have more datatypes available to you. If your program relies on an insertion order, say so. If anyone ever moves to another implementation, they will thank you for it.

Ordered dictionaries already existed as a separate type. Much of the gain was in the use of dicts for core language features. In particular, Python 3.6 guarantees that the order in which class attributes are defined is preserved, and that functions which take variadic keyword arguments receive them in the order in which they were passed. It permits other implementations to use types other than dict to accomplish that…

And this is all the more reason using a type would make sense. Curious why things use ordered? Look at all the places that declare they need ordered. :)

Re: Python dicts are now ordered

#295

Post by thedirt0115 »

MereInterest wrote: post #204
_whiteCaps_ wrote: post #196

Earlier quoted context omitted.

Golang map iteration is returned in random order specifically to make sure that people don't rely on the order. I think this feature says a lot about the philosophy of Python vs Go.

Sounds like a fast and idiomatic way to shuffle a deck of cards is then to convert to a map and back.

Convert a deck of cards ([]Card?) to a map (map[Card]bool?) and back just to shuffle? That's unlikely to be faster or more idiomatic than a straightforward implementation of the Fisher-Yates shuffle[1]. Try writing the code to do it both ways and compare.

[1]: https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle

Re: Python dicts are now ordered

#296

Post by philwelch »

jwkane wrote: post #255
anthonypasq wrote: post #241

Earlier quoted context omitted.

in what reasonable use case would the order of the properties on an object matter? I can't think of one

when you are diffing the serialized output?

From what I recall of the JSON standard itself, there's no guarantee about key ordering being significant. If you're diffing serialized output to compare two JSON objects you need to be serializing it in a consistent format, otherwise even whitespace is going to throw you off.

Re: Python dicts are now ordered

#297

Post by tylerjwilk00 »

skrebbel wrote: post #242
svnpenn wrote: post #226

Earlier quoted context omitted.

> PHP really got this one right (and immediately messed it up by mixing it with arrays, but hey, PHP). No, it didnt. What PHP calls "array" is actually an Ordered Map, as you alluded to: https://yaml.org/type/omap it literally says that in the documentation, paragraph one, sentence one: > An array in PHP is actually an ordered map. https://php.net/types.array The issue, if one exists, is that PHP doesnt have a sequen…

Sure, PHP arrays are actually ordered maps, but it's super confusing when used as arrays. I mean, look at code like this: $arr = []; $arr[2] = "two"; $arr[0] = "zero"; $arr[1] = "one"; echo join($arr, ", "); // two, zero, one In any other language, the result of similar code would be "zero, one, two". You'd need to `ksort` this thing to get it to behave like a normal bog-standard boring array. Everybody's writing PHP…

I have to disagree also.

The order inserted makes most sense in most cases. Hand crafting integer based array indexes out of order is rare. And as you say a simple ksort call guarantees sort order.

    ksort($arr);
    echo join($arr, ", ");
    // one, two, three

Re: Python dicts are now ordered

#298

Post by tene »

skrebbel wrote: post #223

This is awesome, because the ordered map is the best data structure out there for easy & predictable programming, possibly only barring the array. There's so many cases where it's a benefit for map entries to retain order (and none where it's a problem). PHP really got this one right (and immediately messed it up by mixing ordered maps with arrays into a big soup, but hey, PHP). And so did, JS, sorta-kinda-by-acciden…

Could you explain some examples of what this is useful for? What sort of algorithms or operations do you have in mind where you both want insertion order, and also key-based lookup in the same data structure?

I've made heavy use of all kinds of maps, and of queues and channels and arrays, but I don't recall ever noticing a situation where I wanted the properties of both mixed into the same data structure.

I'd love to learn more about useful tools to add to my toolbox!

Re: Python dicts are now ordered

#299

Post by onei »

stestagg wrote: post #206
onei wrote: post #202

Earlier quoted context omitted.

I was under the impression python dict ordering was deterministic, just not in an order recognisable by humans, i.e. ordered by hashes. Was this not the case?

From the docs: "CPython implementation detail: Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary’s history of insertions and deletions." So I would claim that equates to non-deterministic

I read that as: if two dicts are constructed in the same way and use the same python implementation then they'd be the same. If you change the way the dicts were generated or the underlying implementation then they're not the same.

To me, nondeterministic is when the input does not change but the output does. The docs essentially say to not rely on the order, not that the result is not reproducible.

Re: Python dicts are now ordered

#300

Post by yvdriess »

rplnt wrote: post #251
BiteCode_dev wrote: post #227

I live in the Python bubble, so I haven't realized until this post that so few people knew about that. This post is massively popular despite talking about a feature we had since Python 3.6, in 2016, that was posted on HN at the time and that is featured in most popular tutorials. A good reminder that most of the world doesn't revolve about my favorite language. And that information is not that fast to spread.

> about a feature we had since Python 3.6, in 2016 No, you had that feature in one implementation. Now it's in the language specification. That's vastly different because only now you can rely on it without fearing it can go away with the next release.

There is no language spec for Python, the CPython implementation defines the language.
Post reply on HN