Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python dicts are now ordered

softwaremaniacs.org

Post reply on HN
Print view View original

271–280 of 457 posts

Re: Python dicts are now ordered

#271

Post by lonelappde »

jakear wrote: post #184
gnulinux wrote: post #142

Earlier quoted context omitted.

Why does it matter if it's explicit or not. If python doesn't support forward compatibility, you should know that if you write code for 3.7, it's not gonna work in 3.5. Doesn't seem like a big deal to me.

If you’re the only one writing it and you’re the only one running it, it’s probably fine. But if I’m putting a file out there that will only work in 3.7, it’d be nice if any potential users of that file would get a good error message if they try to run it on 3.5, rather than wrong results. I could potentially assert a version, but do I really want to do that each time I wrote something that might be used somewhere el…

If you write code that doesn't work in 3.5, you should check the version at startup and exit

Re: Python dicts are now ordered

#272

Post by skrebbel »

inopinatus wrote: post #268
skrebbel wrote: post #223

This is awesome, because the ordered map is the best data structure out there for easy & predictable programming, possibly only barring the array. There's so many cases where it's a benefit for map entries to retain order (and none where it's a problem). PHP really got this one right (and immediately messed it up by mixing ordered maps with arrays into a big soup, but hey, PHP). And so did, JS, sorta-kinda-by-acciden…

Point of order: Ruby’s hashes do preserve insertion order. Python dicts are now the same.

Good for Ruby! Nice to hear.

I'm pretty sure they didn't in Ruby ~1.8 though. It's been awhile :-) Can't remember exactly which version, but it bit me more than once so I'm pretty sure :)

Re: Python dicts are now ordered

#273

Post by masklinn »

onei wrote: post #202
_aleph2c_ wrote: post #144

This is an amazing contribution to the language. A mixture of speed and convenience, probably made by volunteers. As for people criticizing a change to what use to be a non-deterministic ordering of a dict iteration; I don't know what to say to them, other than, are you serious? There are people out there who are working for us, they work for free and they did some heavy lifting to give us this. They might read what…

I was under the impression python dict ordering was deterministic, just not in an order recognisable by humans, i.e. ordered by hashes. Was this not the case?

> Was this not the case?

From the origin to Python 3.2, iteration order was arbitrary but deterministic (in CPython, it was not guaranteed to be determinstic in other implementations)

From Python 3.3 to Python 3.5, iteration order was non-determinstic (across runs, it was deterministic per-process instance)

In Python 3.6 it's unspecified but deterministic and follows insertion order (in CPython)

In Python 3.7, Python 3.6's behaviour was specified and documented

Re: Python dicts are now ordered

#274

Post by visarga »

peteradio wrote: post #87
setr wrote: post #51

Earlier quoted context omitted.

Sure, but you can't safely take everything from a higher version to a lower version in any case; if insertion order became gauranteed due to a bugfix, and wasn't backported, you'd be in the same boat. The only way to consistently code cross-version is to start with the lowest you plan to support (assuming the higher versions are actually backwards-compatible). Does any language gaurantee that code is both backwards a…

Issue seems to be silent incorrect behavior, what happens if you attempt to run python code containing f-strings using an older python version. Does it raise an exception? That's good! What happens now if you write code for 3.7 which takes advantage of the new ordering and someone grabs it from your repo and runs it using 3.2, it would happily give incorrect results and noone is the wiser.

If you expect this situation you can assert the language version.

Re: Python dicts are now ordered

#275

Post by skrebbel »

ebg13 wrote: post #269
svnpenn wrote: post #252

Earlier quoted context omitted.

...thats the same thing. Maybe youre thinking about "sorted maps"?

It's not the same thing. Compare two OrderedDicts that weren't constructed in the same order and the comparison will fail. Compare two dicts that weren't constructed in the same order and the comparison will pass. The difference is whether order is part of its identity. A dict is still just a set of pairs, not a list of them. It just happens that it also guarantees now that if you _iterate_ over the set you'll walk t…

Good point, but close enough! My points about how it makes the data structure more awesome stands.

FWIW I don't think people agree on whether "element order is part of the value's identity" is part of the definition of "ordered map". There's plenty other languages & libs with ordered maps where the order isn't part of its identity but they still use the term "ordered map" in the name or the docs. Eg PHP's arrays or ImmutableJS's OrderedMap.

Re: Python dicts are now ordered

#276

Post by masklinn »

the_matrix wrote: post #189
_aleph2c_ wrote: post #144

This is an amazing contribution to the language. A mixture of speed and convenience, probably made by volunteers. As for people criticizing a change to what use to be a non-deterministic ordering of a dict iteration; I don't know what to say to them, other than, are you serious? There are people out there who are working for us, they work for free and they did some heavy lifting to give us this. They might read what…

I don't have any criticism if the performance of this new dict stays the same as earlier and that's a huge if. Considering python3 is already a crawling language compared to peers like java and php, I think devs should focus more on that before handing out features.

> I don't have any criticism if the performance of this new dict stays the same as earlier and that's a huge if.

1. it's been there since 2016, the new dict was released as part of Python 3.6 (though the ordering was only an implementation detail until 3.7)

2. the new dict is much more compact (20~25%) and should have much faster iteration speed, those were actually the original goals, ordered iteration was a side-effect

3. the new dict should not benchmark significantly slower across the board, it wouldn't have been merged if it were a regression

Re: Python dicts are now ordered

#277

Post by TheFlyingFish »

yurlungur wrote: post #195
hprotagonist wrote: post #47

Earlier quoted context omitted.

If you know you're supporting old code, use OrderedDict. you arguably ought to anyway, for explicitness.

Yeah I think this is probably my main issue. I don't think it's reasonable to ask users of your code to always use 3.7+ instead of 3.6 if they are usually expected to be compatible. And it's also unnecessary to break such compatibility for something like preferring dict over OrderedDict anyways. At least I would try to avoid any such issues by still using OrderedDict. That said, I have no idea about the internals of…

It actually improves performance. Or at least, it comes along with a set of performance improvements that give you ordering for free. Raymond Hettinger has a great talk on it: https://www.youtube.com/watch?v=npw4s1QTmPg&t=1s

Re: Python dicts are now ordered

#278

Post by kingdomcome50 »

skrebbel wrote: post #242
svnpenn wrote: post #226

Earlier quoted context omitted.

> PHP really got this one right (and immediately messed it up by mixing it with arrays, but hey, PHP). No, it didnt. What PHP calls "array" is actually an Ordered Map, as you alluded to: https://yaml.org/type/omap it literally says that in the documentation, paragraph one, sentence one: > An array in PHP is actually an ordered map. https://php.net/types.array The issue, if one exists, is that PHP doesnt have a sequen…

Sure, PHP arrays are actually ordered maps, but it's super confusing when used as arrays. I mean, look at code like this: $arr = []; $arr[2] = "two"; $arr[0] = "zero"; $arr[1] = "one"; echo join($arr, ", "); // two, zero, one In any other language, the result of similar code would be "zero, one, two". You'd need to `ksort` this thing to get it to behave like a normal bog-standard boring array. Everybody's writing PHP…

I disagree. The behavior you exhibit makes more sense than the behavior you imagine.

You start with an _empty_ array, then (somehow) set the third element of that array? That makes very little sense.

Had you _initialized_ your example as an array with a length of 3, your desired behavior would, indeed, manifest.

Re: Python dicts are now ordered

#279

Post by d4lt »

styfle wrote: post #244
skrebbel wrote: post #223

This is awesome, because the ordered map is the best data structure out there for easy & predictable programming, possibly only barring the array. There's so many cases where it's a benefit for map entries to retain order (and none where it's a problem). PHP really got this one right (and immediately messed it up by mixing ordered maps with arrays into a big soup, but hey, PHP). And so did, JS, sorta-kinda-by-acciden…

Also don’t forget about JS Map which is ordered https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Obligatory reminder: https://bugs.chromium.org/p/v8/issues/detail?id=164

Re: Python dicts are now ordered

#280

Post by masklinn »

jmilloy wrote: post #212
mark-r wrote: post #94

Earlier quoted context omitted.

All they have to do is give it a different name, such as unordered_dict.

I might have preferred odict be added and dict left alone, so that you didn't have to import OrderedDict when really you just want the new dict. If dict and odict mapped to the same thing for a while, fine, but dict could diverge again if desired.

The problem is that when you behave a certain predictable and deterministic way users will leverage that even if it's not specified.

That is actually why the Python maintainers decided to make this behaviour official: the ordering was the consequence of changes in implementation details, but over 3.6's lifecycle they feared it would cause compatibility issues as users would start relying on the ordering properties of CPython and that would be an issue for alternate implementations (though pypy had switched to the same implementation even earlier so was also insertion-ordered even when running in Python 2.7 mode).

Providing stronger guarantees was considered useful and unlikely to be severely detrimental in the long run, so the project decided to make it part of the language spec.

Post reply on HN