Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python dicts are now ordered

softwaremaniacs.org

Post reply on HN
Print view View original

101–110 of 457 posts

Re: Python dicts are now ordered

#101

Post by Biganon »

riazrizvi wrote: post #68
jkbbwr wrote: post #15

Am I the only one that thinks this is a stupid decision? This will silently break code that starts to rely on this behaviour that gets executed on Python3.5 and lower. I would consider changing how a builtin works to be a major breaking change. It would have been fine if this was a change between 2 and 3 but on a minor version? Thats insane.

That is an original line of thinking in the world of software, “Let’s not add a feature to a new version because it wouldn’t work if used in an older version”.

It would silently introduce bugs, that's the whole issue.

Re: Python dicts are now ordered

#102

Post by [deleted] »

nobleach wrote: post #9
pfdietz wrote: post #4

Earlier quoted context omitted.

Is the ordering there at least fixed, or can it vary from run to run even on the same data, or between implementations or versions of the same implementation? EDIT: to clarify, I was asking about JavaScript there.

It is deterministic, but it's an odd rule-set. For example: `var o = { 3: 'foo', 1: 'bar', b: 'baz', a: 'quux' }` Will yield: `{1: "bar", 3: "foo", b: "baz", a: "quux"}`. Numeric keys get sorted. String keys are insertion-order. If key order is a priority, a Map should be used instead. Often when JavaScript's objects are used and a particular sort order is required, an accompanying (sorted) array is used.

[deleted]

Re: Python dicts are now ordered

#103

Post by VWWHFSfQ »

abetlen wrote: post #91
jhrmnn wrote: post #57

Earlier quoted context omitted.

What you describe is forward compatibility, and Python (and most other programming language) doesn't have it.

TIL I suppose Python doesn't even have backwards compatibility within the same major release as we saw with the addition of the async keyword in Python3.5. Many older Python 3 packages broke because they expected that to be a legal identifier for a variable name.

Python has never tried to maintain backwards compatibility within a major release

Re: Python dicts are now ordered

#104

Post by kstrauser »

japhyr wrote: post #5

Brandon Rhodes gave a great talk at PyCon 2017 about the evolution of dictionaries in Python. I really appreciate the people who dive deep into these implementation details, and continue to optimize the languages we all use. https://www.youtube.com/watch?v=66P5FMkWoVU

I usually feel like a reasonably bright person, and then I go to a talk like this and suddenly remember how terribly much I don't (and will never) know. There are some very sharp people at play here.

Re: Python dicts are now ordered

#105

Post by TheNorthman »

tele_ski wrote: post #86
rob74 wrote: post #84

Earlier quoted context omitted.

The Go designers went the other way (as they often do): > When iterating over a map with a range loop, the iteration order is not specified and is not guaranteed to be the same from one iteration to the next. Actually it is not only "not guaranteed to be the same", the runtime actively makes sure that the iteration order is actually different so you don't even start to rely on it...

That seems like it would be a performance hit to actively make it different? That seems like a very weird and strange design decision if true

It actually doesn't make that much of a difference. See this commit for actual benchmarks about the impact: https://github.com/golang/go/commit/3be4d95731a17073afb1f69b...

Re: Python dicts are now ordered

#106

Post by daenz »

Cool, but, this just encourages people to rely on hashmaps being ordered by insertion. The nature of a hashmap is not one where you rely on the ordering. This is going to trip up newbies who then move to other languages.

Some comments on the post don't seem to understand the changes, for example, the C++ comment saying that std::map is ordered, so this isn't a strange change. The difference is that std::map is a red-black tree implementation, which has a traversal order, not a hashmap with insertion order. So there is already confusion.

Re: Python dicts are now ordered

#107

Post by bootlooped »

6gvONxR4sf7o wrote: post #98

I'd love to know more about where an ordered dict comes in handy. Anyone have use cases? Otherwise, guaranteeing this behavior just seems ‾\_(ツ)_/‾ If this is useful, I wonder if an ordered set is useful.

I've used TreeMap in Java many times. I don't recall exactly why I used it, I think at least some of the cases were simply that it was convenient to be able to store an ordered list of two different types without having to go through the trouble of making a full blown class that holds those types.

Re: Python dicts are now ordered

#108

Post by masklinn »

andybak wrote: post #95
biddlesby wrote: post #26

Earlier quoted context omitted.

It doesn’t break existing code. Code written for Python 3.7 might break on older versions of Python

What versions of Python didn't have this behaviour? It was there but just not guaranteed. I don't see how anything could break (unless there are alternative implementations with different behaviour I'm not aware of?)

> It was there but just not guaranteed.

Dicts have been effectively ordered since 3.6. Iteration order was literally randomised (at process start) before 3.6. I'm also not sure whether the behaviour under deletion was changed between 3.6 and 3.7 so it's possible that there are subtle differences there.

Re: Python dicts are now ordered

#109

Post by zurtex »

rcfox wrote: post #70
xapata wrote: post #66

Earlier quoted context omitted.

OrderedDict is less efficient. It's best to replace usage with basic dict where possible to improve efficiency. There is one obscure feature of OrderedDict that isn't in the basic, so it can't always be swapped.

Really? I would have figured they'd just do something like: class OrderedDict(dict): pass (Plus a little bit of API shimming.)

No, OrderedDict has it's own C implementation which was created just before it was decided that dict would preserve order across iteration.

Further there is a big difference, regular dict preserves order across iteration but OrderedDict treats order up to equality.

I.e. this returns True:

    {1: 1, 2: 2} == {2: 2, 1: 1}
Where as this returns False:

    OrderedDict({1: 1, 2: 2}) == OrderedDict({2: 2, 1: 1})
To make that difference speedy it needs to be done on the C level.

Re: Python dicts are now ordered

#110

Post by dallbee »

tele_ski wrote: post #86
rob74 wrote: post #84

Earlier quoted context omitted.

The Go designers went the other way (as they often do): > When iterating over a map with a range loop, the iteration order is not specified and is not guaranteed to be the same from one iteration to the next. Actually it is not only "not guaranteed to be the same", the runtime actively makes sure that the iteration order is actually different so you don't even start to rely on it...

That seems like it would be a performance hit to actively make it different? That seems like a very weird and strange design decision if true

It is a performance hit and I initially had the same reservations. Importantly, the implementation imposes a small overhead on the access time of iteration over a map, but not in a way which changes the scaling behavior. Other map operations are not implicated.

I think the decision was wise - it prevents user code from relying on ordered iteration behavior, which allows the go team to switch to different map implementations without fear of breaking user code.

If the order had been unspecified, but iteration was still ordered in practice, there'd undoubtedly be (incorrect) user code that relied on that behavior.

Amusingly, I now run into engineers who are under the misconception that map iteration is random, and proceed to use maps as an RNG. That's an unfortunate mistake because Go's map iteration is quite non-uniform - it's shuffled just enough to appear random to the untrained eye, while be performant.

Post reply on HN