Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PEP 810 – Explicit lazy imports

peps.python.org

Post reply on HN
Print view View original

171–180 of 247 posts

Re: PEP 810 – Explicit lazy imports

#171

Post by pjjpo »

I wonder if this proposal suffers an because of Python's extremely generous support period and perhaps the ship has sailed.

- lazy imports are a hugely impactful feature

- lazy imports are already possible without syntax

This means any libraries that get large benefit from lazy imports already use import statements within functions. They can't really use the new feature since 3.14 EoL is _2030_, forever from now. The __lazy_modules__ syntax preserves compatibility only but being eager, not performance - libraries that need lazy imports can't use it until 2030.

This means that the primary target for a long time is CLI authors, which can have a more strict python target and is mentioned many times in the PEP, and libraries that don't have broad Python version support (meaning not just works but works well), which indicates they are probably not major libraries.

Unless the feature gets backported to 2+ versions, it feels not so compelling. But given how it modifies the interpreter to a reasonable degree, I wonder if even any backport is on the table.

Re: PEP 810 – Explicit lazy imports

#172

Post by shiandow »

I don't quite understand why they forbid this from being used in functions.

I mean I get why that makes the most sense in most scenarios, but it is not as if the problem of having to choose between declaring dependencies up front or deferring expensive imports until needed does not happen in functions.

Take for instance a function that quickly fails because the arguments are incorrect, it might do a whole bunch of imports that only make sense for that function but which are made immediately obsolete.

It feels like it is forbidden just because someone thought it wasn't a good coding style but to me there is no obvious reason it couldn't work.

Re: PEP 810 – Explicit lazy imports

#173

Post by simonw »

pests wrote: post #160
simonw wrote: post #108

Earlier quoted context omitted.

Yea, that's the core problem here: plugins can add new CLI subcommands, which means they all need to be loaded on startup. https://llm.datasette.io/en/stable/plugins/plugin-hooks.html...

Could you cache the help doc after first full loaded run and only regenerate when new plugins are added / updated?

It's not just help - the plugins need to be imported so the root level CLI tool knows what to do if you type "llm subcommand ..." where that subcommand is defined by a plugin.

Re: PEP 810 – Explicit lazy imports

#174

Post by Inufu »

simonw wrote: post #173
pests wrote: post #160

Earlier quoted context omitted.

Could you cache the help doc after first full loaded run and only regenerate when new plugins are added / updated?

It's not just help - the plugins need to be imported so the root level CLI tool knows what to do if you type "llm subcommand ..." where that subcommand is defined by a plugin.

In that case the CLI only needs to import the plugin that defined that sub command, not all plugins?

Re: PEP 810 – Explicit lazy imports

#175

Post by oskarkk »

zahlman wrote: post #103
BugsJustFindMe wrote: post #71

Earlier quoted context omitted.

> It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. You're making the common mistake of conflating how things currently work with how things could work if the responsible group agrees to…

No, changing this breaks the world. A huge fraction of PyPI becomes completely invalid overnight, and the rest fails the expected version checks. Not to mention that the language is fundamentally designed around the expectation that modules are singleton. I've written about this at length before but I can't easily find it right now (I have way too many bookmarks and not nearly enough idea how to organize them). Yes,…

> Not to mention that the language is fundamentally designed around the expectation that modules are singleton.

Modules being singletons is not a problem in itself I think? This could work like having two versions of the same library in two modules named like library_1_23 and library_1_24. In my program I could hypothetically have imports like `import library_1_23 as library` in one file, and `import library_1_24 as library` in another file. Both versions would be singletons. Then writing `import library==1.23` could be working like syntax sugar for `import library_1_23 as library`.

Of course, having two different versions of a library running in the same program could be a nightmare, so all of that may not be a good idea at all, but maybe not because of module singletons.

Re: PEP 810 – Explicit lazy imports

#176

Post by Grikbdl »

pjjpo wrote: post #171

I wonder if this proposal suffers an because of Python's extremely generous support period and perhaps the ship has sailed. - lazy imports are a hugely impactful feature - lazy imports are already possible without syntax This means any libraries that get large benefit from lazy imports already use import statements within functions. They can't really use the new feature since 3.14 EoL is _2030_, forever from now. The…

In at least the scientific python environment, there's "SPEC0" in which a lot of the de facto core libraries have basically agreed to support the last three versions of Python, no more.

For other libraries they can of course choose as they want, but generally I don't think it's so common for libraries to be as generous with the support length as cpython.

Re: PEP 810 – Explicit lazy imports

#177

Post by jacquesm »

flare_blitz wrote: post #146
jacquesm wrote: post #138

Earlier quoted context omitted.

In spite of the 'You're welcome to bring' this does not actually sound like an encouragement but more of a veiled statement that some non-technical reason will be found to shoot down the proposal if it were to be made so you might as well not bother.

No, the point is that most people in this thread do not appreciate the complexity of implementing lazy imports. If you disagree, your energy is better spent talking to a CPython core developer about implementation details of making baseless assertions from an ivory tower. There are many people here who think enabling lazy imports is as simple as flipping a light switch. They have no idea what they're talking about.

Reading comprehension failure on your part.

Re: PEP 810 – Explicit lazy imports

#178

Post by fastasucan »

gorgoiler wrote: post #105

I don’t really agree with the premise: > The dominant convention in Python code is to place all imports at the beginning of the file. This avoids repetition, makes import dependencies clear and minimizes runtime overhead. > A somewhat common way to delay imports is to move the imports into functions, but this practice requires more work [and] obfuscates the full set of dependencies for a module. The first part is jus…

>The second part is basically saying if you do your own logic-based lazy imports (inline imports in functions) then you’re going against the traditions.

Think you need to read again.

Re: PEP 810 – Explicit lazy imports

#179

Post by simonw »

Inufu wrote: post #174
simonw wrote: post #173

Earlier quoted context omitted.

It's not just help - the plugins need to be imported so the root level CLI tool knows what to do if you type "llm subcommand ..." where that subcommand is defined by a plugin.

In that case the CLI only needs to import the plugin that defined that sub command, not all plugins?

It doesn't know which plugin defines a subcommand until it imports the plugin's module.

I'm happy with the solution I have now, which is to encourage plugin authors not to import PyTorch or other heavy dependencies at the root level of their plugin code.

Post reply on HN