Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating to Python 3 with pleasure

github.com

Post reply on HN
Print view View original

21–30 of 181 posts

Re: Migrating to Python 3 with pleasure

#21

Post by lgas »

metaobject wrote: post #20
jstimpfle wrote: post #11

Earlier quoted context omitted.

> test_path = datasets_root / dataset / 'test' > Previously it was always tempting to use string concatenation (concise, but obviously bad), now with pathlib the code is safe, concise, and readable. This is the kind of feature that I'm wary to use even in scripts: questionable benefit, and probably too clever.

I don't have a machine available right now, but I wonder what happens if two adjacent path elements are integers? Does it perform division instead of path/string concatenation?

  TypeError: expected str, bytes or os.PathLike object, not int

Re: Migrating to Python 3 with pleasure

#22

Post by thijsvandien »

rtpg wrote: post #19
StavrosK wrote: post #12

Earlier quoted context omitted.

No, you're right. I believe dict orders were made deterministic (by insertion order) in 3.6. Before that, it was undefined.

CPython interates in this fashion since 3.6, but it is not part of the Python spec. Might be a bit contentious but it's not supposed to be relied upon

It was recently decided that as of 3.7 it will be.

(Source: https://mail.python.org/pipermail/python-dev/2017-December/1...)

Re: Migrating to Python 3 with pleasure

#24

Post by jstimpfle »

lgas wrote: post #21
metaobject wrote: post #20

Earlier quoted context omitted.

I don't have a machine available right now, but I wonder what happens if two adjacent path elements are integers? Does it perform division instead of path/string concatenation?

TypeError: expected str, bytes or os.PathLike object, not int

This probably means that you put a path to the left, not ints on both sides.

Re: Migrating to Python 3 with pleasure

#26

Post by fyi1183 »

jstimpfle wrote: post #16
VHRanger wrote: post #13

Earlier quoted context omitted.

Dicts are UNORDERED associative containers. If youre depending your app on implementation defined behavior, that's on your developpers shoulders. Stuff like that shouldn't pass code review

It is surprising though that the order was non-deterministic, not only undefined. Determinism is something I intuitively expect from read-only operations like iteration.

Non-determinism comes from randomness in the hash function. I don't know the details of the Python implementation, but generally you want randomness at the very least when processing potentially hostile user input. Otherwise your code is susceptible to a denial of service attack where the attacker sends data that was crafted to maximize hash collisions, to break the expected performance of hash tables.

From a theoretical angle, the only way to build hash tables with some kind of provable runtime guarantees is to include randomness.

Re: Migrating to Python 3 with pleasure

#27

Post by brett40324 »

jstimpfle wrote: post #16
VHRanger wrote: post #13

Earlier quoted context omitted.

Dicts are UNORDERED associative containers. If youre depending your app on implementation defined behavior, that's on your developpers shoulders. Stuff like that shouldn't pass code review

It is surprising though that the order was non-deterministic, not only undefined. Determinism is something I intuitively expect from read-only operations like iteration.

It is simply that a python dict is not a list (read array).

Re: Migrating to Python 3 with pleasure

#28

Post by StavrosK »

jstimpfle wrote: post #16
VHRanger wrote: post #13

Earlier quoted context omitted.

Dicts are UNORDERED associative containers. If youre depending your app on implementation defined behavior, that's on your developpers shoulders. Stuff like that shouldn't pass code review

It is surprising though that the order was non-deterministic, not only undefined. Determinism is something I intuitively expect from read-only operations like iteration.

I think that's a habit you need to shake. If something is undefined, you should intuitively shy away from expecting things from it. If I found myself wondering why some undefined behavior was not consistent, I'd question why I even believed it should be consistent in the first place.

Re: Migrating to Python 3 with pleasure

#29

Post by Avshalom »

djrobstep wrote: post #17
VHRanger wrote: post #13

Earlier quoted context omitted.

Dicts are UNORDERED associative containers. If youre depending your app on implementation defined behavior, that's on your developpers shoulders. Stuff like that shouldn't pass code review

Not for much longer, as of 3.7 the ordering is a language feature: https://mail.python.org/pipermail/python-dev/2017-December/1... It's mad that it ever wasn't this way. Mapping-with-ordered-keys is such a useful and pervasive data structure (all database query result rows, for one) that an ordered dictionary should be a fundamental part of a language. It has been so much more pleasant to write python since ordering…

>>(all database query result rows, for one)

What? No. SQL does not return results in any consistent ordering unless specifically instructed to.

Re: Migrating to Python 3 with pleasure

#30

Post by tomlock »

minimaxir wrote: post #3

I'm not sure why this is targeted toward data scientists; these tips are useful for any Python user.

I think it's because the remaining python 2.7 users are mostly people using the packages that were unsupported in 3, which I think were mostly packages used in data science afaik
Post reply on HN