How the Python import system works • Hacker News Board

Re: How the Python import system works

#101

Post by kbenson » Sat, Jul 24, 2021, 7:19 PM UTC

yongjik wrote: post #90

Probably half of the commenters here know this, but since we're here, this is my go-to boilerplate for starting a python script. (Probably won't work on Windows.) #!/bin/sh # Run the interpreter with -u so that stdout isn't buffered. "exec" "python3" "-u" "$0" "$@" import os import sys curdir = os.path.dirname(os.path.realpath(sys.argv[0])) # Add enough .. to point to the top-level project directory. sys.path.insert(…

There's no well known module to do most this for you? In perl, the recent canonical way is to use the FindBin module to find the current binary's running dir, and the the local::lib module to set the import path (or just use lib for older style library dirs). That always seemed cumbersome to me at 2-3 lines that weren't very clean looking.

Also, say what you will about Perl and esoteric global variables, but it's kinda nice to be able to toggle buffered output on and off on the fly. Is there really no way to do this in python without re-executing the script like that?

Re: How the Python import system works

#102

Post by ledauphin » Sat, Jul 24, 2021, 7:20 PM UTC

hirowan wrote: post #58

I'm just grateful I'm not a core Python dev after reading this thread. I've never seen so much negativity concentrated in one place in quite some time, for a feature of a programming language which is fairly innocuous.

Python is the language everyone at HN loves to hate. One presumes it has something to do with the fact that it's Y Combinator's recommendation for most startups.

Re: How the Python import system works

#103

Post by blakesley » Sat, Jul 24, 2021, 7:20 PM UTC

captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Earnest question: why are you all trying to use relative imports? What problem is that solving for you? I've never even bothered to try it out because it seems potentially problematic in the way all relative references can be, e.g., relative file paths.

Re: How the Python import system works

#104

Post by trulyme » Sat, Jul 24, 2021, 7:21 PM UTC

brilee wrote: post #25

johncena33 wrote: post #3

I've been programming in Python professionally for more than five years. I consider myself quite a good programmer. I still don't have good grasp on Python's import system. Does anyone else have similar experience?

I'm a python readability approver at Google and I don't understand how the import system works

Interesting... If someone wants to know more about what a readability approver does: https://www.pullrequest.com/blog/google-code-review-readabil...

Re: How the Python import system works

#105

Post by whoknowswhat11 » Sat, Jul 24, 2021, 7:21 PM UTC

captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

I never followed the changes around imports in 3 … but def still run into issues especially trying to move code from dev to deploy

Re: How the Python import system works

#106

Post by trulyme » Sat, Jul 24, 2021, 7:26 PM UTC

geofft wrote: post #46

fullstop wrote: post #21

Earlier quoted context omitted.

I sometimes fall into the trap of using pip to install dependencies and then things break after an os update. That is, my python version has changed from 3.8 to 3.9 and my dependencies are sitting in the wrong directory. I never know if I should use pip and requirements.txt or rely on Ubuntu's packaged versions.

I used to have a rule of, never use pip and only use Ubuntu's/Debian's packaged versions. That works pretty well if you're happy with the packaged versions and you don't need unpackaged libraries. I now have the rule of, only ever use pip inside a venv . If your venv is more than a little bit complex, write a requirements.txt file so you can generate it. So it's something like $ cat > requirements.txt .gitignore $ py…

For me the rule is to always use pipenv locally and pip + requirements.txt (generated by pipenv) for production (in docker container usually). No complaints.

Re: How the Python import system works

#107

Post by icegreentea2 » Sat, Jul 24, 2021, 7:31 PM UTC

kbenson wrote: post #101

yongjik wrote: post #90

Probably half of the commenters here know this, but since we're here, this is my go-to boilerplate for starting a python script. (Probably won't work on Windows.) #!/bin/sh # Run the interpreter with -u so that stdout isn't buffered. "exec" "python3" "-u" "$0" "$@" import os import sys curdir = os.path.dirname(os.path.realpath(sys.argv[0])) # Add enough .. to point to the top-level project directory. sys.path.insert(…

There's no well known module to do most this for you? In perl, the recent canonical way is to use the FindBin module to find the current binary's running dir, and the the local::lib module to set the import path (or just use lib for older style library dirs). That always seemed cumbersome to me at 2-3 lines that weren't very clean looking. Also, say what you will about Perl and esoteric global variables, but it's kin…

Ya... if you're trying to get the path of the script, you can use `__file__` special variable (instead of loading it from bash $0 and grabbing sys.argv[0]).

For adding current directory to front of path, the sys.path.insert() call is a pretty sound way of doing it.

Re: How the Python import system works

#108

Post by The_Colonel » Sat, Jul 24, 2021, 7:38 PM UTC

roenxi wrote: post #19

But despite being terrible the Python import system is remarkably easy to get started in. And generally easy for beginners to work with (put all the code in the same folder or "pip install"). There are some lessons here that other languages would do well to learn. Trouble importing 3rd party libraries must be a kiss of death for beginner engagement.

I vividly remember the frustration when trying to make some very simple application imports working. There's a couple of gotchas, the biggest one being that you need to write imports based on where you expect to run your applications from. Perhaps obvious for experienced python devs, very much surprising for newcomers.

And then I was quite shocked by the state of package managers in python. You need to learn pip, venv (with references to "virtualenv"), these are too low level, so you find pipenv, which is unbelievably slow (installing a single dependency can take 10 minutes), so you need to learn to use it with "--skip-lock", but then you lose locking ...

I've never appreciated node's bundled "npm" so much before which mostly "just works".

Re: How the Python import system works

#109

Post by perlgeek » Sat, Jul 24, 2021, 7:38 PM UTC

chrisseaton wrote: post #7

MeteorMarc wrote: post #4

Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.

> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?

A Customer has a BillingContact, which references a Person, which has primary Customer.

Boom, circular dependency.

Happens in basically all corporate code bases that grow over the years, with varying path lengths.

Throwing all potentially circular types into one big module isn't a great solution.

(In practice, we tend to rely on run-time imports to make it work. Not really great, but better than throwing several 10k or 100k lines of code into a single module).

Re: How the Python import system works

#110

Post by rcxdude » Sat, Jul 24, 2021, 7:38 PM UTC

toxik wrote: post #72

grawprog wrote: post #71

Years ago, I thought I'd try learning Python, I'd heard it was supposed to be easy, good for beginners and everything. I read one of those beginner Python type books and followed along with a roguelike tutorial. Everything was going pretty alright, until I started trying to split everything into different files and use imports. I ended up just giving up. I read programming in lua, and rewrote my entire project in lua…

Strange, Python’s import is not very difficult to use.

It has some wildly frustratingly unintuitive behaviours in precisely the wrong place for beginners: in between having everything in a single script and building a proper package, especially when you are invoking your script with 'python script.py' as opposed to say 'python -m scripts.script'.