How the Python import system works
11–20 of 209 posts
Re: How the Python import system works
#12Badly
Re: How the Python import system works
#13Slightly off-topic:
One of the simplest import systems I've seen were in q/kdb (like with most things in that language, everything is as simple as possible)
Imports work by simply calling `\l my_script.q` which is similar to simply taking the file `my_script.q` and running it line by line (iirc, it does so eagerly, so it reruns the entire file whenever you do `\l my_script.q`, even if the file has been loaded before, which may affect you state. By contrast, Python `import` statements are no-ops if the module has already been imported).
The main disadvantage is that you risk your imported script overwriting your global variables. This is solved by following the strong (unenforced) conversion that scripts should only affect their own namespaces (which works by having the script write declare \d .my_namespace at the top of the script)
I never found this system limiting and always appreciated its simplicity - whenever things go wrong debugging is fairly easy.
What does Python gain by having a more sophisticated import system?
Re: How the Python import system works
#14I've been programming in Python professionally for more than five years. I consider myself quite a good programmer. I still don't have good grasp on Python's import system. Does anyone else have similar experience?
The average python programmer does not really need to deal with pythons import system that much (just be aware of how it does its module loading and that you can conditionally do stuff sometimes with __import__ etc). As someone who has messed around with it a lot (dynamically loading/unloading modules, modifying on the fly etc) I would NOT recommend doing that stuff for anything in production.
Re: How the Python import system works
#15Slightly off-topic: One of the simplest import systems I've seen were in q/kdb (like with most things in that language, everything is as simple as possible) Imports work by simply calling `\l my_script.q` which is similar to simply taking the file `my_script.q` and running it line by line (iirc, it does so eagerly, so it reruns the entire file whenever you do `\l my_script.q`, even if the file has been loaded before,…
Everything is an object, and it “just works” for most purposes. (Not nearly as many as I would like, but still; it has backwards-compatibility to consider, so I'll cut it some slack.)
If you need to start installing your own user packages, you need `pip` and then `venv` and then things get ugly, but for the usual case where the sysadmin deals with all that (or you're on Windows), it works quite well.
Re: How the Python import system works
#16Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.
> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?
I don't think purposeful circular dependency, but you can end up with circular imports after refactoring for instance.
The common approach to solving this is pulling everything that is used by all the modules into leaf libraries, effectively creating a directed acyclic graph, but this is not obvious nor easy to do the first time.
Re: How the Python import system works
#17Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.
> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?
In my experience they creep in over time as the system grows. Coupling between parts of the system that was previously unnecessary is added, and the cycles form.
Re: How the Python import system works
#18Slightly off-topic: One of the simplest import systems I've seen were in q/kdb (like with most things in that language, everything is as simple as possible) Imports work by simply calling `\l my_script.q` which is similar to simply taking the file `my_script.q` and running it line by line (iirc, it does so eagerly, so it reruns the entire file whenever you do `\l my_script.q`, even if the file has been loaded before,…
Python imports aren't necessarily always importing Python source code -- they can be pyc (bytecode) files, or C API extensions, etc.
These are slightly more complicated than "load the script at this path".
There's probably a more detailed answer, in that historically, decisions were made that we're now stuck with. Python packages can and sometimes intentionally have import-time side effects, for example. They must be only run once, without relying on convention, or we break existing code.
Re: How the Python import system works
#19But despite being terrible the Python import system is remarkably easy to get started in. And generally easy for beginners to work with (put all the code in the same folder or "pip install").
There are some lessons here that other languages would do well to learn. Trouble importing 3rd party libraries must be a kiss of death for beginner engagement.
Re: How the Python import system works
#20Python importing quirks can be time consuming. Things beginners will encounter: - three modules cannot depend on each other in a circular way - relative imports are fragile ("module not found") - the __all__ definitions in the __init__ file make modules available under different full names - how to reload a module in a jupyter notebook if edited and so on.
> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?
The purpose is you want to use code in other modules.
If you keep doing that for a while, a circular dependency will happen.
This is the dumbest thing thing in Python. All other languages I know have solved it.