Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PEP 810 – Explicit lazy imports

peps.python.org

Post reply on HN
Print view View original

81–90 of 247 posts

Re: PEP 810 – Explicit lazy imports

#81

Post by BiteCode_dev »

Alir3z4 wrote: post #31
f311a wrote: post #26

I don't like the idea of introducing a new keyword. We need a backward compatible solution. I feel like Python needs some kind of universal annotation syntax such as in go (comments) or in Rust (macros). New keyword means all parsers, lsps, editors should be updated. I’m pretty sure there will be new keywords in Python in the future that only solve one thing.

Me neither. Introducing new keyword has become a recent thing in Python. Seems Python has a deep scare since Python2 to Python3 time and is scared to do anything that causes such drama again. For me, the worst of all is "async". If 2to3 didn't cause much division, the async definitely divided Python libraries in 2. Sync and Async. Maybe if they want backward compatible solution, this can be done by some compile or ru…

Async was a soft keyword for many, many years in order to maintain compat. And before that, asyncio used yield to be even more compatible.

They took a decade to solidify that. At some point, you have to balance evolution and stability. For a language as popular as Python, you can not break the world every week, but you can't stay put for 5 years.

Re: PEP 810 – Explicit lazy imports

#82

Post by driggs »

zahlman wrote: post #60
dheera wrote: post #45

Earlier quoted context omitted.

Oof. I wish they could support version imports import torch==2.6.0+cu124 import numpy>=1.2.6 and support having multiple simultaneous versions of any Python library installed. End this conda/virtualenv/docker/bazel/[pick your poison] mess

It's been explained many times before why this is not possible: the library doesn't actually have a version number. The distribution of source code on PyPI has a version number, but the name of this is not connected to the name of any module or package you import in the source code. The distribution can validly define zero or more modules (packages are a subset of modules, represented using the same type in the Pytho…

> It's been explained many times before why this is not possible: the library doesn't actually have a version number.

Not possible? Come on.

Almost everyone already uses one of a small handfull of conventional ways to specify it, eg `__version__` attribute. It's long overdue that this be standardized so library versions can reliably be introspected at runtime.

Allowing multiple versions to be installed side-by-side and imported explicitly would be a massive improvement.

Re: PEP 810 – Explicit lazy imports

#83

Post by peterfirefly »

simonw wrote: post #32

Love this. My https://llm.datasette.io/ CLI tool supports plugins, and people were complaining about really slow start times even for commands like "llm --help" - it turned out there were popular plugins that did things like import pytorch at the base level, so the entire startup was blocked on heavy imports. I ended up adding a note to the plugin author docs suggesting lazy loading inside of functions - https://llm.…

Parse the command line and do things like "--help" without doing the imports.

Only do imports when you know you need them -- or as an easy approximation, only if the easy command line options have been handled and there's still something to do.

Re: PEP 810 – Explicit lazy imports

#84

Post by BiteCode_dev »

TheMrZZ wrote: post #2

Feels like a good feature, with a simple explanation, real world use cases, and a scoped solution (global only, pretty simple keyword). I like it!

Agree, they really did their homework, listed edge cases, made practical compromises, chose not to overdo it, reworked it again and again quite a bit and compared it to real life experience.

It's really beautiful work, especially since touching the back bone (the import system) of a language as popular as Python with such a diverse community is super dangerous surgery.

I'm impressed.

Re: PEP 810 – Explicit lazy imports

#85

Post by syncsynchalt »

Spivak wrote: post #75
comex wrote: post #67

I don't hate it but I don't love it. It sounds like everyone will start writing `lazy` before essentially every single import, with rare exceptions where eager importing is actually needed. That makes Python code visually noisier. And with no plan to ever change the default, the noise will stay forever. I would have preferred a system where modules opt in to being lazy-loaded, with no extra syntax on the import side.…

I don't think this makes sense to be on the module side, the caller is the one with the information as to whether the module can or needs to be lazily loaded. There's nothing really for the module being imported to decide, every module can be lazily loaded. Even if it has side effects the caller may want to defer those as well.

I think side-effects are exactly the problem, you can't have the runtime default to lazy-loading all modules without breaking code that e.g. relies on side effects running before thread creation or forking.

Re: PEP 810 – Explicit lazy imports

#86

Post by the__alchemist »

colpabar wrote: post #14
the__alchemist wrote: post #13

Does this fix the circular imports problem that comes up if you don't structural your programs in a hierarchical way?

This is what I thought of too. I really only know python, do other languages not have that issue? In python it does not seem like a "problem" to me - whenever I have seen circular import issues it is because the code is organized poorly. I worry that this feature will lead to devs "fixing" circular import issues by using lazy imports.

Interesting. I find that I always have this problem in any non-trivial python project, and don't find this to be due to poorly organized code. I have only seen this requirement in Python.

IME circular import errors aren't due to poor organization; they're due to an arbitrary restriction Python has.

Re: PEP 810 – Explicit lazy imports

#87

Post by veber-alex »

comex wrote: post #67

I don't hate it but I don't love it. It sounds like everyone will start writing `lazy` before essentially every single import, with rare exceptions where eager importing is actually needed. That makes Python code visually noisier. And with no plan to ever change the default, the noise will stay forever. I would have preferred a system where modules opt in to being lazy-loaded, with no extra syntax on the import side.…

I would gladly take a command line flag that I can pass to python that makes all module loading lazy.

Unless you are writing scripts or very simple stuff running side effects when modules are loaded should be avoided at all cost anyway.

Re: PEP 810 – Explicit lazy imports

#88

Post by zahlman »

IshKebab wrote: post #8

Wake me up when we can import a module by relative file path.

Relative imports have been supported for approximately forever (https://stackoverflow.com/questions/72852). If you mean "by explicitly specifying a path string" (as opposed to a symbolic name), that has also been supported for approximately forever (https://stackoverflow.com/questions/67631). Today, the `importlib` standard library exposes all the steps of the import process — including figuring out where the source code is, checking the `sys.modules` cache etc. — and lets you hook into everything (in particular, you can bypass the entire "finder" process and turn a file path into a dummy "spec" which is fed to a "loader").

The flexibility of this system also entails that you can in effect define a completely new programming language, describe the process of creating Python bytecode from your custom source, and have clients transparently `import` source in the other language as a result. Or you can define an import process that grabs code from the Internet (not that it would be a good idea...).

If you mean "by explicitly specifying a relative path, and having it be interpreted according to the path of the current module's source code", well first you have to consider that the current module isn't required to have source code. But if it does, then generally it will have been loaded with the default mechanism, which means the module object will have a `__file__` attribute with an absolute path, and you just set your path relative to that.

Re: PEP 810 – Explicit lazy imports

#89

Post by zahlman »

Spivak wrote: post #19
IshKebab wrote: post #8

Wake me up when we can import a module by relative file path.

You… can? I mean in the strictest sense you're technically not importing by file path but if you make your folder a module by slapping an __init__.py in there then your relative imports will follow the directory tree. I think as of Python 3.3 the init file is optional so it will do it by default but I can't remember if there are still some cases where it's required. The only thing you can't do is go "up" to a higher…

> if you make your folder a module by slapping an __init__.py in there then your relative imports will follow the directory tree.

`__init__.py` has nothing to do with making this work. It is neither necessary (as of 3.3, yes, you got it right: see https://peps.python.org/pep-0420/) nor sufficient (careless use of sys.path and absolute imports could make it so that the current folder hasn't been imported yet, so you can't even go "up into" it). The folder will already be represented by a module object.

What `__init__.py` does is:

1. Prevents relative imports from also potentially checking in other paths.

2. Provides a space for code that runs before sub-modules, for example to set useful package attributes such as `__all__` (which controls star-imports).

Re: PEP 810 – Explicit lazy imports

#90

Post by lyu07282 »

jmward01 wrote: post #43

This is needed, but I don't like new keywords. What I would love, for many reasons, is if we could decorate statements. Then things like: import expensive_module could be: @lazy import expensive_module or you could do: @retry(3) x = failure_prone_call(y) lazy is needed, but maybe there is a more basic change that could give more power with more organic syntax, and not create a new keyword that is special purpose (and…

Would this statement decorator then manipulate the AST of the following statement or how would that work?
Post reply on HN