Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the Python import system works

tenthousandmeters.com

Post reply on HN
Print view View original

111–120 of 209 posts

Re: How the Python import system works

#111

Post by Waterluvian »

brundolf wrote: post #32
Waterluvian wrote: post #10

Earlier quoted context omitted.

I’m always amazed just how tolerant javascript’s import system is when I have circular imports. I guess maybe because it doesn’t care about modules and just cares about specific elements that are being imported/exported. When I do have a nasty circular dependency Webpack usually does a bad job telling me what’s wrong. Though I should still treat circular imports as, at the very least, an organization code smell.

Circular imports in JS matter when the imported code is being called immediately at import time. If a file defines functions that later call functions from another file (and vice-versa), but those symbols will be populated before the function actually gets called, there's no problem

Yeah. It’s so flexible. I get frustrated at Python (Django) serializers that legitimately need to depend on each other. And the answer on the forums is to create a near duplicate class.

Re: How the Python import system works

#112

Post by nemetroid »

lp251 wrote: post #59
chrisseaton wrote: post #55

Earlier quoted context omitted.

> Now you have a circular import problem; glhf But why not collapse into a single module at this point if you can’t avoid dependency? What are the separate modules adding at this stage forward?

You can, but sometimes it’s not ideal. I ran into this when A and B had many derived classes. I wanted to put A and it’s derived classes in one module, and B and it’s derived classes in another. It was messy. I wound up putting A and B in a single module and having a separate one for the derived classes. Not ideal.

It does sound ideal to me, or at least better than the initial proposal.

A and B both need to know about the other's base definition, neither cares about the details about the other's derived classes. Splitting it into three modules shares as little surface area as possible.

Re: How the Python import system works

#113

Post by AlphaSite »

The_Colonel wrote: post #108
roenxi wrote: post #19

But despite being terrible the Python import system is remarkably easy to get started in. And generally easy for beginners to work with (put all the code in the same folder or "pip install"). There are some lessons here that other languages would do well to learn. Trouble importing 3rd party libraries must be a kiss of death for beginner engagement.

I vividly remember the frustration when trying to make some very simple application imports working. There's a couple of gotchas, the biggest one being that you need to write imports based on where you expect to run your applications from. Perhaps obvious for experienced python devs, very much surprising for newcomers. And then I was quite shocked by the state of package managers in python. You need to learn pip, ven…

Poetry is pretty great comparatively, since it handles both dependencies, locking and virtual environments for you, but it has slow resolution just like pip (since the new update), pyenv, pipenv, etc.

Re: How the Python import system works

#114

Post by AlphaSite »

btilly wrote: post #28
maest wrote: post #13

Slightly off-topic: One of the simplest import systems I've seen were in q/kdb (like with most things in that language, everything is as simple as possible) Imports work by simply calling `\l my_script.q` which is similar to simply taking the file `my_script.q` and running it line by line (iirc, it does so eagerly, so it reruns the entire file whenever you do `\l my_script.q`, even if the file has been loaded before,…

Python avoids having to rerun the import over and over again. Suppose that you are importing a larger project. Where your one import (say of your standard database interface) pulls in a group of modules (say one for each table in your database), all of which import a couple of base libraries (to make your database objects provide a common interface) that themselves pull in common Python modules (like psychopg2) which…

Yep. Rerunning things is how you end up with C/C++ style headers and all the (performance and otherwise) problems there in.

Re: How the Python import system works

#115

Post by [deleted] »

perlgeek wrote: post #109
chrisseaton wrote: post #7

Earlier quoted context omitted.

> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?

A Customer has a BillingContact, which references a Person, which has primary Customer. Boom, circular dependency. Happens in basically all corporate code bases that grow over the years, with varying path lengths. Throwing all potentially circular types into one big module isn't a great solution. (In practice, we tend to rely on run-time imports to make it work. Not really great, but better than throwing several 10k…

[deleted]

Re: How the Python import system works

#116

Post by chrisseaton »

perlgeek wrote: post #109
chrisseaton wrote: post #7

Earlier quoted context omitted.

> three modules cannot depend on each other in a circular way What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?

A Customer has a BillingContact, which references a Person, which has primary Customer. Boom, circular dependency. Happens in basically all corporate code bases that grow over the years, with varying path lengths. Throwing all potentially circular types into one big module isn't a great solution. (In practice, we tend to rely on run-time imports to make it work. Not really great, but better than throwing several 10k…

I guess I don’t get why so you want to use separate modules if you aren’t getting the benefits of modularisation?

Re: How the Python import system works

#117

Post by yongjik »

icegreentea2 wrote: post #107
kbenson wrote: post #101

Earlier quoted context omitted.

There's no well known module to do most this for you? In perl, the recent canonical way is to use the FindBin module to find the current binary's running dir, and the the local::lib module to set the import path (or just use lib for older style library dirs). That always seemed cumbersome to me at 2-3 lines that weren't very clean looking. Also, say what you will about Perl and esoteric global variables, but it's kin…

Ya... if you're trying to get the path of the script, you can use `__file__` special variable (instead of loading it from bash $0 and grabbing sys.argv[0]). For adding current directory to front of path, the sys.path.insert() call is a pretty sound way of doing it.

Yeah, I think you're right - didn't know about __file__.

(To clarify, using "$0" with bash is just standard method to invoke the same script with an interpreter - sys.argv[0] will work with or without bash exec part.)

Re: How the Python import system works

#118

Post by perlgeek »

chrisseaton wrote: post #116
perlgeek wrote: post #109

Earlier quoted context omitted.

A Customer has a BillingContact, which references a Person, which has primary Customer. Boom, circular dependency. Happens in basically all corporate code bases that grow over the years, with varying path lengths. Throwing all potentially circular types into one big module isn't a great solution. (In practice, we tend to rely on run-time imports to make it work. Not really great, but better than throwing several 10k…

I guess I don’t get why so you want to use separate modules if you aren’t getting the benefits of modularisation?

I still get some benefits of modularisation, even if there are some cross-dependencies.

Re: How the Python import system works

#119

Post by aasasd »

catlifeonmars wrote: post #78
sillysaurusx wrote: post #27

Fun fact: you can overload the Python import system to work with other languages that you create . I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py import foo checks for "foo.l", and if found, compiles it to foo.py on the fly and imports that instead. It's so cursed. I love it.

NodeJS allows this as well. I think this is pretty much a must-have feature for any serious dynamic language. Edit: A must have for any prolific dynamic language. But now I’m not sure that’s true, because even though it apparently works in Python, it’s certainly not widely used. In NodeJS this feature is used quite heavily for typescript, coffeescript (etcetera) interop.

It's pretty much the exact feature that's behind the saying that ‘to parse Perl you must have the Perl interpreter’. Because Perl allows some kind of language/imports handlers, as exhibited by tricks like Lingua::Romana::Perligata.

Re: How the Python import system works

#120

Post by bobbylarrybobby »

captainmuon wrote: post #45

There are two things that are annoying about Python's import system. Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...). Number two is, in order to do package man…

Always an informative (and fun) read:

https://stackoverflow.com/questions/14132789/relative-import...

Post reply on HN