Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the Python import system works

tenthousandmeters.com

Post reply on HN
Print view View original

181–190 of 209 posts

Re: How the Python import system works

#181

Post by andrewaylett »

andrewaylett wrote: post #138
sillysaurusx wrote: post #27

Fun fact: you can overload the Python import system to work with other languages that you create . I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py import foo checks for "foo.l", and if found, compiles it to foo.py on the fly and imports that instead. It's so cursed. I love it.

Lots of things you can do with Python but probably shouldn't and people typically don't. That's one reason I prefer it to Ruby, or even Node, where monkey-patching or otherwise exposing bad magical behaviours is common and even encouraged -- the power is all there, but the ecosystem encourages you to use it for good, not evil. This sounds very much like the good kind of magic, though.

To add to this: bad magic is magic that the user has to be aware of in order to use safely. Good magic is an implementation detail that the user doesn't need to know anything about.

Re: How the Python import system works

#182

Post by masklinn »

kdmccormick wrote: post #151
bobbylarrybobby wrote: post #120

Earlier quoted context omitted.

Always an informative (and fun) read: https://stackoverflow.com/questions/14132789/relative-import...

Nice, do you know of anything similarly-comprehensive that is updated for Python 3? IIRC, Python 3 simplified things a good deal by removing "implicit" relative imports, but I'm a little foggy on exactly what that means.

> IIRC, Python 3 simplified things a good deal by removing "implicit" relative imports, but I'm a little foggy on exactly what that means.

Exactly that: in Python 3, if an import doesn’t start with a . it will always be absolute aka looked up from sys.path.

In python 2, it would try performing a relative import (so importing a sibling module) first.

Re: How the Python import system works

#183

Post by [deleted] »

codethief wrote: post #148
xavdid wrote: post #85

Earlier quoted context omitted.

Once I wrapped my head around when you can and can't use relative imports, I've been pretty ok with them. The think that irks me is whether they work changes based on where you've invoked Python from. `./bin/my_script.py` behaves differently from `./my_script.py`. Coming from JS, that was a pretty frustrating realization.

> The think that irks me is whether they work changes based on where you've invoked Python from. `./bin/my_script.py` behaves differently from `./my_script.py`. This. IMO that's also one of the main issues of Bash – you can't modularize a script unless you make sure your working directory is the directory containing the script. (And good luck with finding out the latter! [0]) [0]: https://stackoverflow.com/questions/…

[deleted]

Re: How the Python import system works

#184

Post by masklinn »

robochat wrote: post #91
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Relative imports used to work much more naturally IMHO in python2 but then they broke it in python3 because Guido wanted scripts and modules to always be separate codebases. So, whereas it used to be easy to have a module that could also be run as a script inside a package, this is now very difficult to implement. To the extent that any python2 code that does this, should probably be refactored when being ported to p…

Are you sure you're not confusing changes to the import system with changes to the default sys.path (especially when executing a file)?

Because unless I'm missing something, the changes to the import system make this more reliable, not less: in Python 2, implicitly relative imports means `import x` might depend on the file's location. In Python 3, it does not, it depends solely on the setup of the sys.path.

Re: How the Python import system works

#185

Post by masklinn »

xavdid wrote: post #85
captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Once I wrapped my head around when you can and can't use relative imports, I've been pretty ok with them. The think that irks me is whether they work changes based on where you've invoked Python from. `./bin/my_script.py` behaves differently from `./my_script.py`. Coming from JS, that was a pretty frustrating realization.

> The think that irks me is whether they work changes based on where you've invoked Python from. `./bin/my_script.py` behaves differently from `./my_script.py`.

It does not though, unless you have altered the default sys.path to always contains `.`.

When running a file / script, Python will add that file's directory as the first lookup path, so these two invocations should have the exact same sys.path, and thus the same module resolution throughout[0].

If you have added `.` to sys.path (which is not the default), then the first invocation will also have the "grand-parent" folder on the sys.path, while the second won't.

This also doesn't seem to have anything to do with relative imports, you need to already be inside a package for relative imports to do anything.

[0] that is not the case of `-c` and `-m`, both add CWD to the path, and they differ in what they do: `-c` adds `''` to sys.path, which is the in-process CWD. `-m` stores the actual value of CWD at invocation, so changes to CWD while the program runs don't influence module resolution

Re: How the Python import system works

#186

Post by pnt12 »

awestroke wrote: post #36
roenxi wrote: post #19

But despite being terrible the Python import system is remarkably easy to get started in. And generally easy for beginners to work with (put all the code in the same folder or "pip install"). There are some lessons here that other languages would do well to learn. Trouble importing 3rd party libraries must be a kiss of death for beginner engagement.

I disagree. I find it hard to get started in python. There are loads of package managers, so I don't know which one to pick. There are multiple different rules for for how imports work with local files. The standard library is full of of functions that one should not use any more, and you need to know which is which. Defining a main entrypoint consists of checking a magic __NAME__ constant. You have internalised all…

Counterpoints:

Start with pip, the official package manager. You don't have to start with virtual envs: I didn't.

I haven't ran into any function which I "should not use".

You don't need an entry point, you can just write code in a file. Besides that, I don't see how much it differs from other languages with implicit entry points, where you need to match a certain name in your function.

Imports: yes, those are annoying and confusing. I still struggle with those.

Aditionally: package managers and virtual envs are a pain in the ass. Every year we're getting a new one which is supposed to solve the problems from the previous one, but doesn't, and the cycle goes on. The language should really solve this at the core instead of requiring community fixing, as it is a core part of any serious development.

Re: How the Python import system works

#187

Post by BossingAround »

zarzavat wrote: post #141
scaryclam wrote: post #132

Earlier quoted context omitted.

So much this. There's no good reason to use relative imports. They're less readable, more dangerous and don't solve a single problem.

In JS land everything is a relative import. It's nice because you can move a whole directory of code from one project to another (or around in the same project) and it still works because all the imports pointing to other files inside that directory were relative, you only have to fix imports that go outside the directory. Also the way that JS imports are just relative paths is very nice because it means that the imp…

> It's nice because you can move a whole directory of code from one project to another (or around in the same project) and it still works

Yes, just like absolute imports...

> relative paths is very nice because it means that the imports are statically determinable, your editor can understand them and fix them automatically and you can trust that refactoring

1) Just like absolute imports?

2) I absolutely contest that relative imports are easier for an IDE to refactor. I've never had VSCode hang while refactoring a Java package; I've had VSCode hang while refactoring "create-react-app" app...

Re: How the Python import system works

#188

Post by jmholla »

johncena33 wrote: post #3

I've been programming in Python professionally for more than five years. I consider myself quite a good programmer. I still don't have good grasp on Python's import system. Does anyone else have similar experience?

I've actually found the Python import documentation [0] to be really approachable. I'd give it a look if this article didn't clear things up.

[0]: https://docs.python.org/3/reference/import.html

Re: How the Python import system works

#189

Post by paulddraper »

GregJJ wrote: post #37

Python import system is by far the worst one I dealt with. Using Setup.py and regular or namespace packages, relative import, having complex sub packages and cross importing, running a script from somewhere inside one of your sub packages, and many more craps like these. Import system must be intuitive and easy to use!

What import system does not have these problems?

Re: How the Python import system works

#190

Post by paulddraper »

Waterluvian wrote: post #10
MeteorMarc wrote: post #4

Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.

I’m always amazed just how tolerant javascript’s import system is when I have circular imports. I guess maybe because it doesn’t care about modules and just cares about specific elements that are being imported/exported. When I do have a nasty circular dependency Webpack usually does a bad job telling me what’s wrong. Though I should still treat circular imports as, at the very least, an organization code smell.

Allowing circular imports was the ugliest part of ECMAScript modules.
Post reply on HN