Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the Python import system works

tenthousandmeters.com

Post reply on HN
Print view View original

151–160 of 209 posts

Re: How the Python import system works

#151

Post by kdmccormick »

bobbylarrybobby wrote: post #120
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Always an informative (and fun) read: https://stackoverflow.com/questions/14132789/relative-import...

Nice, do you know of anything similarly-comprehensive that is updated for Python 3? IIRC, Python 3 simplified things a good deal by removing "implicit" relative imports, but I'm a little foggy on exactly what that means.

Re: How the Python import system works

#152

Post by true_religion »

noptd wrote: post #147
zarzavat wrote: post #141

Earlier quoted context omitted.

In JS land everything is a relative import. It's nice because you can move a whole directory of code from one project to another (or around in the same project) and it still works because all the imports pointing to other files inside that directory were relative, you only have to fix imports that go outside the directory. Also the way that JS imports are just relative paths is very nice because it means that the imp…

Given most editors handle absolute imports as good as if not better than relative imports, I don't see any real benefit to using the latter.

It stops some circular imports if you use relative imports.

Re: How the Python import system works

#153

Post by jnwatson »

sillysaurusx wrote: post #27

Fun fact: you can overload the Python import system to work with other languages that you create . I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py import foo checks for "foo.l", and if found, compiles it to foo.py on the fly and imports that instead. It's so cursed. I love it.

See also https://github.com/drathier/stack-overflow-import for stupid Python import tricks.

"from stackoverflow import quick_sort will go through the search results of [python] quick sort looking for the largest code block that doesn’t syntax error in the highest voted answer from the highest voted question and return it as a module. If that answer doesn’t have any valid python code, it checks the next highest voted answer for code blocks."

I once implemented a custom importer as part of a system where the Python interpreter never touched the filesystem.

Re: How the Python import system works

#154

Post by Waterluvian »

nyrikki wrote: post #126
Waterluvian wrote: post #111

Earlier quoted context omitted.

Yeah. It’s so flexible. I get frustrated at Python (Django) serializers that legitimately need to depend on each other. And the answer on the forums is to create a near duplicate class.

JavaScript does not provide namespaces by default which allows this. Python is built upon namespaces and cycles in dependencies, either viewed as graph theory or kSAT introduce that fun NP-complete problem of version hell. Using `need` in Javascript maintains the directed acyclic graph structure, but if you get fancy you will run into the same problems with circular depends in Python. Karp's 21 and/or SAT will catch…

Sorry what’s ‘need’ ? I can’t seem to find it as a keyword.

Re: How the Python import system works

#155

Post by bredren »

Carl Meyer, a Python developer at Instagram, has ~recently discussed open source contributions to Python he has worked on.

He specifically describes “strict modules” and efforts to improve the efficiency of Python imports at 16:30 of this episode of Django Chat:

https://django-chat.simplecast.com/episodes/django-instagram...

Re: How the Python import system works

#156

Post by heavyset_go »

captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

> Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...).

I think the thing to remember about using relative imports is that it requires modules. Using relative imports to import something into a script will fail because the scripts don't belong to modules.

> Number two is, in order to do package management, you have to create a fake python installation and bend PYTHONPATH. Virtualenvs are the canonical way to do it, but to me it feels like a hack - the core language seems to wants all packages installed in /usr. So now I have all these virtualenvs lying around and they are detached from the scripts.

This is a pain. You can abstract this away with with pyenv or virtualenvwrapper, though.

Re: How the Python import system works

#157

Post by maleldil »

BurningFrog wrote: post #98
chrisseaton wrote: post #63

Earlier quoted context omitted.

> The purpose is you want to use code in other modules. So put them in the same module? Circular modules don’t give you the benefits of modules, do they? Not an expert in modularity.

Putting all your code in one file does indeed solve all import problems, but creates far bigger ones. In case you didn't know, each source code file is a module in Python.

If you have A -> B -> C -> A, then you need all three of A, B and C to be defined if you're going to use any one of those modules. When that's the case, the only thing you're gaining by using modules is the organisation of the file.

Re: How the Python import system works

#158

Post by not_skynet »

xavdid wrote: post #85
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Once I wrapped my head around when you can and can't use relative imports, I've been pretty ok with them. The think that irks me is whether they work changes based on where you've invoked Python from. `./bin/my_script.py` behaves differently from `./my_script.py`. Coming from JS, that was a pretty frustrating realization.

I actually spent time wrestling with exactly this earlier today, and I think I finally have a 2 line solution for it:

https://pastebin.com/5WCXb2Gg

Assuming `foo.bar` is a script inside the package `foo` that you want to both import and run directly (without `python -m ...`), this lets you do so without too much hassle.

Re: How the Python import system works

#159

Post by np_tedious »

true_religion wrote: post #152
noptd wrote: post #147

Earlier quoted context omitted.

Given most editors handle absolute imports as good as if not better than relative imports, I don't see any real benefit to using the latter.

It stops some circular imports if you use relative imports.

I'd rather suffer the late import with absolutes than wrestle with relative imports

Re: How the Python import system works

#160

Post by neoncontrails »

jnwatson wrote: post #153
sillysaurusx wrote: post #27

Fun fact: you can overload the Python import system to work with other languages that you create . I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py import foo checks for "foo.l", and if found, compiles it to foo.py on the fly and imports that instead. It's so cursed. I love it.

See also https://github.com/drathier/stack-overflow-import for stupid Python import tricks. "from stackoverflow import quick_sort will go through the search results of [python] quick sort looking for the largest code block that doesn’t syntax error in the highest voted answer from the highest voted question and return it as a module. If that answer doesn’t have any valid python code, it checks the next highest voted…

That stackoverflow library is so gloriously bonkers. Is it useful in practice?
Post reply on HN