PEP 810 – Explicit lazy imports
131–140 of 247 posts
Re: PEP 810 – Explicit lazy imports
#132Earlier quoted context omitted.
Parse the command line and do things like "--help" without doing the imports. Only do imports when you know you need them -- or as an easy approximation, only if the easy command line options have been handled and there's still something to do.
Well yes, or you can just use the `lazy` keyword, when it makes it into core.
Re: PEP 810 – Explicit lazy imports
#133Earlier quoted context omitted.
Oof. I wish they could support version imports import torch==2.6.0+cu124 import numpy>=1.2.6 and support having multiple simultaneous versions of any Python library installed. End this conda/virtualenv/docker/bazel/[pick your poison] mess
It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. The distribution can validly define zero or more modules (packages are a subset of modules, represented using the same type in the Pytho…
import numpy==2.1
And let's say numpy didn't expose a version number in a standard (which could be agreed upon in a PEP) field, then it would just throw an import exception. It wouldn't break any old code. And only packages with that explicit field would support the pinned version import.
And it wouldn't involve trying to extract and parse versions from older packages with some super spotty heuristics.
But it would make new code impossible to use with older versions of python, and older packages, but that's already the case.
Maybe the issue is with module name spacing?
Re: PEP 810 – Explicit lazy imports
#134Earlier quoted context omitted.
Relative imports have been supported for approximately forever ( https://stackoverflow.com/questions/72852 ). If you mean "by explicitly specifying a path string" (as opposed to a symbolic name), that has also been supported for approximately forever ( https://stackoverflow.com/questions/67631 ). Today, the `importlib` standard library exposes all the steps of the import process — including figuring out where the sou…
Nope. Relative imports work by relative package path, which is not at all the same. Often when you run Python you don't even have a package path. Using `importlib` is a horrible hack that breaks basically all tooling. You very very obviously are not supposed to do that.
This is an assertion that has absolutely no reasoning behind it. I'm not saying I disagree; I'm just saying there is a time and a place for importlib.
Re: PEP 810 – Explicit lazy imports
#135Lazy imports have been proposed before, and were rejected most recently back in 2022: https://discuss.python.org/t/pep-690-lazy-imports-again/1966... . If I recall correctly, lazy imports are a feature supported in Cinder, Meta's version of CPython, and the PEP was driven by folks that worked on Cinder. Last time, a lot of the discussion centered around questions like: Should this be opt-in or opt-out? At what level?…
This is not fearmongering. There is a reason why the only flavor of Python with lazy imports comes from Meta, which is one of the most well-resourced companies in the world.
Too many people in this thread hold the view of "importing {pandas, numpy, my weird module that is more tangled than an eight-player game of Twister} takes too long and I will gladly support anything that makes them faster". I would be willing to bet a large sum of money that most people who hold this opinion are unable to describe how Python's import system works, let alone describe how to implement lazy imports.
PEP 690 describes a number of drawbacks. For example, lazy imports break code that uses decorators to add functions to a central registry. This behavior is crucial for Dash, a popular library for building frontends that has been around for more than a decade. At import-time, Dash uses decorators to bind a JavaScript-based interface to callbacks written in Python. If these imports were made lazy, Dash would break. Frontends used by thousands, if not millions of people, would immediately become unresponsive.
You may cry, "But lazy imports are opt-in! Developers can choose to opt-out of lazy imports if it doesn't work for them." What if these imports were transitive? What if our frontend needed to be completely initialized before starting a critical process, else it would cause a production outage? What if you were a maintainer of a library that was used by millions of people? How could you be sure that adding lazy imports wouldn't break any code downstream? Many people made this argument for type hints, which is sensible because type hints have no effect on runtime behavior*. This is not true for lazy imports; import statements exist in essentially every nontrivial Python program, and changing them to be lazy will fundamentally alter runtime behavior.
This is before we even get to the rest of the issues the PEP describes, which are even weirder and crazier than this. This is a far more difficult undertaking than many people realize.
---
* You can make a program change its behavior based on type annotations, but you'd need to explicitly call into typing APIs to do this. Discussion about this is beyond the scope of this post.
Re: PEP 810 – Explicit lazy imports
#136Re: PEP 810 – Explicit lazy imports
#137Earlier quoted context omitted.
You're welcome to bring a concrete proposal to, e.g., https://discuss.python.org/c/ideas/6 , or ask around the core devs to find a PEP sponsor. Note that you will be expected to have familiarized yourself generally with previous failed proposals of this sort, and proactively considered all the reasonably obvious corner cases.
Tbh you’re just reinforcing mine and others point there that the issue isn’t a technical one.
Re: PEP 810 – Explicit lazy imports
#138Earlier quoted context omitted.
The hard part is making the change. Adding an escape hatch so older code still works is easy in comparison. Nobody is claiming this is a trivial problem to solve but its also not an impossible problem. Other languages have managed to figure out how to achieve this and still maintain backwards compatibility.
You're welcome to bring a concrete proposal to, e.g., https://discuss.python.org/c/ideas/6 , or ask around the core devs to find a PEP sponsor. Note that you will be expected to have familiarized yourself generally with previous failed proposals of this sort, and proactively considered all the reasonably obvious corner cases.
Re: PEP 810 – Explicit lazy imports
#139Earlier quoted context omitted.
Oof. I wish they could support version imports import torch==2.6.0+cu124 import numpy>=1.2.6 and support having multiple simultaneous versions of any Python library installed. End this conda/virtualenv/docker/bazel/[pick your poison] mess
It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. The distribution can validly define zero or more modules (packages are a subset of modules, represented using the same type in the Pytho…
That sounds like it is absolutely fixable to me, but more of a matter of not having the will to fix it based on some kind of traditionalism. I've used python, a lot. But it is stuff like this that is just maddeningly broken for no good reason at all that has turned me away from it. So as long as I have any alternative I will avoid python because I've seen way too many accidents on account of stuff like this and many lost nights of debugging only to find out that an easily avoidable issue became - once again - the source of much headscratching.