Earlier quoted context omitted.
Well yes, or you can just use the `lazy` keyword, when it makes it into core.
You can “just” use a feature which does not exist yet? How is that something you “just” do?
lazy from __future__ import __lazy_import__PEP 810 – Explicit lazy imports
191–200 of 247 posts
Re: PEP 810 – Explicit lazy imports
#192Love this. My https://llm.datasette.io/ CLI tool supports plugins, and people were complaining about really slow start times even for commands like "llm --help" - it turned out there were popular plugins that did things like import pytorch at the base level, so the entire startup was blocked on heavy imports. I ended up adding a note to the plugin author docs suggesting lazy loading inside of functions - https://llm.…
You can implement this from your tool today: https://news.ycombinator.com/item?id=45467489 Note that this is global to the entire process, so for example if you make an import of Numpy lazy this way, then so are the imports of all the sub-modules. Meaning that large parts of Numpy might not be imported at all if they aren't needed, but pauses for importing individual modules might be distributed unpredictably across…
Note the PEP does have a FAQ entry that mentions reasons they believe this proposed solution might be preferable to LazyLoader
https://pep-previews--4622.org.readthedocs.build/pep-0810/#f...
Q: Why not use importlib.util.LazyLoader instead?
A: LazyLoader has significant limitations:
Requires verbose setup code for each lazy import.
Has ongoing performance overhead on every attribute access.
Doesn’t work well with from ... import statements.
Less clear and standard than dedicated syntax.
Re: PEP 810 – Explicit lazy imports
#193Feels like a good feature, with a simple explanation, real world use cases, and a scoped solution (global only, pretty simple keyword). I like it!
Hopefully they learned lessons from why PEP-690 was rejected. I've spent quite a while trying to build this stuff for our codebase and it's never worked well enough to use.
690 is mentioned a few times, including a FAQ note on the differences
https://pep-previews--4622.org.readthedocs.build/pep-0810/#f...
Q: How does this differ from the rejected PEP 690?
A: PEP 810 takes an explicit, opt-in approach instead of PEP 690’s implicit global approach. The key differences are:
Explicit syntax: lazy import foo clearly marks which imports are lazy.
Local scope: Laziness only affects the specific import statement, not cascading to dependencies.
Simpler implementation: Uses proxy objects instead of modifying core dictionary behavior.
Re: PEP 810 – Explicit lazy imports
#194Earlier quoted context omitted.
> Using `importlib` is a horrible hack that breaks basically all tooling. You very very obviously are not supposed to do that. This is an assertion that has absolutely no reasoning behind it. I'm not saying I disagree; I'm just saying there is a time and a place for importlib.
Well not if you want high quality Python code. Pylint and Pyright won't understand it, and those are absolutely critical to writing Python code that works reliably.
> those are absolutely critical to writing Python code that works reliably.
Curious how much reliable Python code was written before those tools existed.
For that matter, curious how much was written before the `types` and `typing` standard library modules appeared.
Re: PEP 810 – Explicit lazy imports
#195I wonder if this proposal suffers an because of Python's extremely generous support period and perhaps the ship has sailed. - lazy imports are a hugely impactful feature - lazy imports are already possible without syntax This means any libraries that get large benefit from lazy imports already use import statements within functions. They can't really use the new feature since 3.14 EoL is _2030_, forever from now. The…
[deleted]
Re: PEP 810 – Explicit lazy imports
#196Earlier quoted context omitted.
You can implement this from your tool today: https://news.ycombinator.com/item?id=45467489 Note that this is global to the entire process, so for example if you make an import of Numpy lazy this way, then so are the imports of all the sub-modules. Meaning that large parts of Numpy might not be imported at all if they aren't needed, but pauses for importing individual modules might be distributed unpredictably across…
Note the PEP does have a FAQ entry that mentions reasons they believe this proposed solution might be preferable to LazyLoader https://pep-previews--4622.org.readthedocs.build/pep-0810/#f... Q: Why not use importlib.util.LazyLoader instead? A: LazyLoader has significant limitations: Requires verbose setup code for each lazy import. Has ongoing performance overhead on every attribute access. Doesn’t work well with fro…
Thanks for pointing it out!
> Has ongoing performance overhead on every attribute access.
I would have expected so, but in my testing it seems like the lazy load does some kind of magic to replace the proxy with the real thing. I haven't properly dug into it, though. It appears this point is removed in the live version (https://peps.python.org/pep-0810).
> Doesn’t work well with from ... import statements.
Hmm. The PEP doesn't seem to explain how reification works in this case. Per the above it's a solved problem for modules; I guess for the from-imports it could be made to work essentially the same way. Presumably this involves the proxy holding a reference to the namespace where the import occurred. That probably has a lot to do with restricting the syntax to top level. (Which is the opposite of how we've seen soft keywords used before!)
> Requires verbose setup code for each lazy import.
> Less clear and standard than dedicated syntax.
If you want to use it in a fine-grained way, then sure.
Re: PEP 810 – Explicit lazy imports
#197Earlier quoted context omitted.
> It's been explained many times before why this is not possible: the library doesn't actually have a version number. Not possible ? Come on. Almost everyone already uses one of a small handfull of conventional ways to specify it, eg `__version__` attribute. It's long overdue that this be standardized so library versions can reliably be introspected at runtime. Allowing multiple versions to be installed side-by-side…
I believe the charitable interpretation is that it is not possible without breaking an enormous amount of legacy code. Which does feel close enough to “not possible”. Some situations could be improved by allowing multiple library versions, but this would introduce new headaches elsewhere. I certainly do not want my program to have N copies of numpy, PyTorch, etc because some intermediate library claims to have just-s…
What do you do today to resolve a dependency conflict when an intermediate library has a just-so dependency tree?
The charitable interpretation of this proposed feature is that it would handle this case exactly as well as the current situation, if the situation isn't improved by the feature.
This feature says nothing about the automatic installation of libraries.
This feature is absolutely not about supporting multiple simultaneous versions of a library at runtime.
In the situation you describe, there would have to be a dependency resolution, just like there is when installing the deps for a program today. It would be good enough for me if "first import wins".
Re: PEP 810 – Explicit lazy imports
#198Earlier quoted context omitted.
You're welcome to bring a concrete proposal to, e.g., https://discuss.python.org/c/ideas/6 , or ask around the core devs to find a PEP sponsor. Note that you will be expected to have familiarized yourself generally with previous failed proposals of this sort, and proactively considered all the reasonably obvious corner cases.
In spite of the 'You're welcome to bring' this does not actually sound like an encouragement but more of a veiled statement that some non-technical reason will be found to shoot down the proposal if it were to be made so you might as well not bother.
It's an allusion to the fact that there is a very long history establishing that the problem is not as simple as it sounds, even if you get past the most basic issues, and it's hard to explain it all in a single coherent post.
Re: PEP 810 – Explicit lazy imports
#199Earlier quoted context omitted.
It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. The distribution can validly define zero or more modules (packages are a subset of modules, represented using the same type in the Pytho…
> It's been explained many times before why this is not possible: the library doesn't actually have a version number. That sounds like it is absolutely fixable to me, but more of a matter of not having the will to fix it based on some kind of traditionalism. I've used python, a lot. But it is stuff like this that is just maddeningly broken for no good reason at all that has turned me away from it. So as long as I hav…
> a matter of not having the will to fix it based on some kind of traditionalism
Do you know what happens when Python does summon the will to fix obviously broken things? The Python 2->3 migration happens. (Perl 6 didn't manage any better, either.) Now "Python 3 is the brand" and the idea of version 4 can only ever be entertained as a joke.
Re: PEP 810 – Explicit lazy imports
#200Earlier quoted context omitted.
It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. The distribution can validly define zero or more modules (packages are a subset of modules, represented using the same type in the Pytho…
You could absolutely have this be part of the language in any regard. The question then becomes how does one implement it in a reasonable way. I think every package should have a __version__ property you should be able to call, then you could have versioned imports. In fact there's already many packages already defining __version__ at a package level. https://packaging.python.org/en/latest/discussions/versionin... Ed…
> In fact there's already many packages already defining __version__ at a package level.
This only helps for those that do, and it hasn't been any kind of standard the entire time. But more importantly, that helps only the tiniest possible bit with resolving the "import a specific version" syntax. All it solves is letting the file-based import system know whether it found the right folder for the requested (or worse: "a compatible") version of the importable package. It doesn't solve finding the right one if this one is wrong; it doesn't determine how the different versions of the same package are positioned relative to each other in the environment (so that "finding the right one" can work properly); it doesn't solve provisioning the right version. And most importantly, it doesn't solve what happens when there are multiple requests for different versions of the same module at runtime, which incidentally could happen arbitrarily far apart in time, and also the semantics of the code may depend on the same object being used to represent the module in both places.