Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the Python import system works

tenthousandmeters.com

Post reply on HN
Print view View original

131–140 of 209 posts

Re: How the Python import system works

#131

Post by blooalien »

MeteorMarc wrote: post #4

Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.

I usually deal with this by defining a cell with the actual contents of the class/module code so that I can just re-execute that cell any time I make changes to it. Then I simply copy/paste all the code back into the module.py file once I'm done tweaking it and playing with it. Thus for me Jupyter sort of operates as almost an IDE of sorts.

Re: How the Python import system works

#132

Post by scaryclam »

blakesley wrote: post #103
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Earnest question: why are you all trying to use relative imports? What problem is that solving for you? I've never even bothered to try it out because it seems potentially problematic in the way all relative references can be, e.g., relative file paths.

So much this. There's no good reason to use relative imports. They're less readable, more dangerous and don't solve a single problem.

Re: How the Python import system works

#133

Post by t-writescode »

GregJJ wrote: post #37

Python import system is by far the worst one I dealt with. Using Setup.py and regular or namespace packages, relative import, having complex sub packages and cross importing, running a script from somewhere inside one of your sub packages, and many more craps like these. Import system must be intuitive and easy to use!

… have you tried Rails’ autoloading? Especially when running rake tasks?

Re: How the Python import system works

#134

Post by dragonwriter »

btilly wrote: post #34
chrisseaton wrote: post #7

Earlier quoted context omitted.

> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?

What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it? What is the purpose of modules? You may as well collapse into a single script and the situation would not be any different, would it? I'm not being facetious here. The answer to the second is the answer to the first. A common example might go like this. You have…

> But you just created a circular dependency!

Only if those things not only need to be able to “get to” each other, but also need to know, at compile time, about the concrete implementation of the others.

That's possible to be a real need, but its also something that often happens because of excessive and unnecessary coupling.

Re: How the Python import system works

#135

Post by dragonwriter »

lp251 wrote: post #54
chrisseaton wrote: post #7

Earlier quoted context omitted.

> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?

Suppose you have two classes, A and B. They are sufficiently complex to merit their own modules. Suppose you have some method of A which does something special if it gets an instance of B, and vice versa. Now you have a circular import problem; glhf

> Suppose you have some method of A which does something special if it gets an instance of B.

While that’s in rare circumstances the right thing to do, it's mostly an anti-pattern—you should be taking an object supporting a protocol, with the behavior difference depending on a field or method of (or actually implemented in a method of) that protocol. If you do that, you don't create a dependency on a concrete class that happens to require the special behavior.

Re: How the Python import system works

#136

Post by dagmx »

yongjik wrote: post #90

Probably half of the commenters here know this, but since we're here, this is my go-to boilerplate for starting a python script. (Probably won't work on Windows.) #!/bin/sh # Run the interpreter with -u so that stdout isn't buffered. "exec" "python3" "-u" "$0" "$@" import os import sys curdir = os.path.dirname(os.path.realpath(sys.argv[0])) # Add enough .. to point to the top-level project directory. sys.path.insert(…

Why not just add a shebang , chmod +x and then you're done?

Re: How the Python import system works

#137

Post by rocqua »

robochat wrote: post #91
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Relative imports used to work much more naturally IMHO in python2 but then they broke it in python3 because Guido wanted scripts and modules to always be separate codebases. So, whereas it used to be easy to have a module that could also be run as a script inside a package, this is now very difficult to implement. To the extent that any python2 code that does this, should probably be refactored when being ported to p…

This decision has bit me in the ass so often.

I want to organize my code logically in directories. As a script grows, I want the ability to spin out parts of that file to separate files.

In order to do that in python, I need separate directories between the script and the spun-out functionality. This ends with a script that says "do function from module" and all code being in the module.

Having code in different directories for no reason except "the import system" sucks. How is this supposed to go?

Re: How the Python import system works

#138

Post by andrewaylett »

sillysaurusx wrote: post #27

Fun fact: you can overload the Python import system to work with other languages that you create . I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py import foo checks for "foo.l", and if found, compiles it to foo.py on the fly and imports that instead. It's so cursed. I love it.

Lots of things you can do with Python but probably shouldn't and people typically don't. That's one reason I prefer it to Ruby, or even Node, where monkey-patching or otherwise exposing bad magical behaviours is common and even encouraged -- the power is all there, but the ecosystem encourages you to use it for good, not evil.

This sounds very much like the good kind of magic, though.

Re: How the Python import system works

#139

Post by rocqua »

blakesley wrote: post #103
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Earnest question: why are you all trying to use relative imports? What problem is that solving for you? I've never even bothered to try it out because it seems potentially problematic in the way all relative references can be, e.g., relative file paths.

People have a script run with python, and want to use code in other files.

This is not supported in python. For reasons beyond my understanding, you are supposed to put the script with python, or with the shebang, in a different directory.

Alternatively, you can always use `python -m` to run your code.

Re: How the Python import system works

#140

Post by danpalmer »

yongjik wrote: post #90

Probably half of the commenters here know this, but since we're here, this is my go-to boilerplate for starting a python script. (Probably won't work on Windows.) #!/bin/sh # Run the interpreter with -u so that stdout isn't buffered. "exec" "python3" "-u" "$0" "$@" import os import sys curdir = os.path.dirname(os.path.realpath(sys.argv[0])) # Add enough .. to point to the top-level project directory. sys.path.insert(…

> # Add enough .. to point to the top-level project directory.

This suggests that there is more than one entry point to the Python project?

While I'm sure there are good reasons for this, and while I'm not criticising your instance of this specifically, as a general point of advice I've found this sort of thing to be a bit of an anti-pattern.

Having one entry that handles things like path setup and other global concerns, before delegating out to subcommands or whatever construct works best makes it much easier to keep the whole codebase aligned in many ways.

Django has a system for this and while it has its flaws, it is nice to have it. Using this, on our main Python codebase of ~400k lines, we have a single manual entry point, plus one server entry point. Coordinating things like configuration, importing, and the application startup process, are therefore essentially non-issues for us for almost all development, even though we have a hundred different operations that a dev can run, each of which could have been a separate tool like this.

Post reply on HN