Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PEP 810 – Explicit lazy imports

peps.python.org

Post reply on HN
Print view View original

231–240 of 247 posts

Re: PEP 810 – Explicit lazy imports

#231

Post by tremon »

wmichelin wrote: post #230

This seems unnecessary if we can run imports from functions, right? This feels like a layer of indirection and a potential source for confusion. Rather than implicitly importing a library when the variable is first used, why don't you just explicitly do it? Edit > A somewhat common way to delay imports is to move the imports into functions (inline imports), but this practice requires more work to implement and mainta…

On the contrary, I think "everyone is already doing this in their own way, so we will offer it as a language feature, which brings these extra benefits" is one of the stronger arguments for introducing a language feature.

Re: PEP 810 – Explicit lazy imports

#232

Post by jacquesm »

zahlman wrote: post #199
jacquesm wrote: post #139

Earlier quoted context omitted.

> It's been explained many times before why this is not possible: the library doesn't actually have a version number. That sounds like it is absolutely fixable to me, but more of a matter of not having the will to fix it based on some kind of traditionalism. I've used python, a lot. But it is stuff like this that is just maddeningly broken for no good reason at all that has turned me away from it. So as long as I hav…

> a matter of not having the will to fix it based on some kind of traditionalism Do you know what happens when Python does summon the will to fix obviously broken things? The Python 2->3 migration happens. (Perl 6 didn't manage any better, either.) Now "Python 3 is the brand" and the idea of version 4 can only ever be entertained as a joke.

Yes, good point. Compared to how the Erlang community has handled decades of change Python does not exactly deserve the beauty prize. The lack of forethought - not to be confused with a lot of hot air - on some of these decisions is impressive. I think that the ability to track developments in near realtime is in conflict with that though. If you want your language to be everything to everybody then there will be some broken bones along the way.

Re: PEP 810 – Explicit lazy imports

#233

Post by wsve »

wmichelin wrote: post #230

This seems unnecessary if we can run imports from functions, right? This feels like a layer of indirection and a potential source for confusion. Rather than implicitly importing a library when the variable is first used, why don't you just explicitly do it? Edit > A somewhat common way to delay imports is to move the imports into functions (inline imports), but this practice requires more work to implement and mainta…

> Rather than implicitly importing a library when the variable is first used, why don't you just explicitly do it?

I think it's on you to explain why that's a better approach for everyone's use cases instead of this language feature

Re: PEP 810 – Explicit lazy imports

#235

Post by dijksterhuis »

jmward01 wrote: post #46
jmward01 wrote: post #43

This is needed, but I don't like new keywords. What I would love, for many reasons, is if we could decorate statements. Then things like: import expensive_module could be: @lazy import expensive_module or you could do: @retry(3) x = failure_prone_call(y) lazy is needed, but maybe there is a more basic change that could give more power with more organic syntax, and not create a new keyword that is special purpose (and…

As a side note, It would be great to have a 'preview' for HN comments. I updated the above because I forgot to add line breaks. Sigh. I bet 'preview' would stop a lot of not well thought out comments too.

you can set “delay” in your profile to delay the live posting of a comment after you click the button.

gives me ten minutes to edit (rewrite if i’m honest) before other people see it.

Re: PEP 810 – Explicit lazy imports

#236

Post by thatxliner »

I was looking at PEP 690 and I saw

> A world in which Python only supported imports behaving in a lazy manner would likely be great...we do not envision the Python langauge transitioning to a world where lazy imports are the default...this concept would add complexity to our ecosystem.

Why can't lazy be the default and instead propose an `eager` syntax? The only argument I can imagine is that there's some API that runs a side effect based on importing, but perhaps making it eager for modules with side effects would be a sufficient temporary fix?

Re: PEP 810 – Explicit lazy imports

#237

Post by lenkite »

zahlman wrote: post #209
lenkite wrote: post #76

Earlier quoted context omitted.

Wish pyproject.toml was enhanced to specify lazy loading via regexs.

I don't follow your reasoning. `pyproject.toml` has nothing to do with what happens at runtime. It's about building and packaging the code. It also doesn't say anything related to the modules that will be imported at runtime. It deals in the names of distributions , which are completely independent of `import` statements.

What I mean is that the `lazy` keyword need not be specified in the source Python files. Instead you can mark off a bunch of modules identified by regex as lazy. Perhaps `pyproject.toml` isn't the best place and it should be an independent python file.

Re: PEP 810 – Explicit lazy imports

#238

Post by flare_blitz »

zahlman wrote: post #204
flare_blitz wrote: post #135

Earlier quoted context omitted.

I also hope this proposal succeeds, but I'm not optimistic. This will break tons of code and introduce a slew of footguns. Import statements fundamentally have side effects, and when and how these side effects are applied will cause mysterious breakages that will keep people up for many nights. This is not fearmongering. There is a reason why the only flavor of Python with lazy imports comes from Meta, which is one o…

> This will break tons of code I don't see how. It adds a new, entirely optional syntax using a soft keyword. The semantics of existing code do not change. Yes, yes, you anticipated the objection: > What if these imports were transitive? ... How could you be sure that adding lazy imports wouldn't break any code downstream? I would need to see concrete examples of how this would be a realistic risk in principle. (My g…

I would suggest reading this blog post from Meta, which should resolve your concerns: https://engineering.fb.com/2024/01/18/developer-tools/lazy-i...

Regarding risks in practice:

> Libraries such as PyTorch, Numba, NumPy, and SciPy, among others, did not seamlessly align with the deferred module loading approach. These libraries often rely on import side effects and other patterns that do not play well with Lazy Imports. The order in which Python imports could change or be postponed, often led to side effects failing to register classes, functions, and operations correctly. This required painstaking troubleshooting to identify and address import cycles and discrepancies.

This isn't precisely the scenario I described above, but it is a concrete example of how deferred imports can cause issues that are difficult to debug.

Regarding performance benefits:

> At Meta, the quest for faster model training has yielded an exciting milestone: the adoption of Lazy Imports and the Python Cinder runtime. ... we’ve been able to significantly improve our model training times, as well as our overall developer experience (DevX) by adopting Lazy Imports and the Python Cinder runtime.

Re: PEP 810 – Explicit lazy imports

#239

Post by hnlmorg »

zahlman wrote: post #198
jacquesm wrote: post #138

Earlier quoted context omitted.

In spite of the 'You're welcome to bring' this does not actually sound like an encouragement but more of a veiled statement that some non-technical reason will be found to shoot down the proposal if it were to be made so you might as well not bother.

It's an allusion to the fact that there is a very long history establishing that the problem is not as simple as it sounds, even if you get past the most basic issues, and it's hard to explain it all in a single coherent post.

Nobody suggested it was easy. We were just arguing against your claim that it’s impossible

Saying something is possible isn’t the same as saying something is easy.

If you were to have said “it’s a different problem to solve because of…” then you wouldn’t have had any pushback. But you didn’t. You said it “this is not possible”. And that’s the part that people were disputing.

Re: PEP 810 – Explicit lazy imports

#240

Post by nilslindemann »

nilslindemann wrote: post #122

This is the wrong syntax, comparable to how "u" strings were the wrong syntax and "b" strings are the right syntax. They make that, what should be the default, a special case. Soon, every new code will use "lazy". The long term effect of such changes is a verbose language syntax. They should have had a period where one, if they want lazy imports, has to do "from __future__ import lazy_import". After that period, lazy…

I have posted this Idea on Python Discussions, and Thomnas Wouters replied to it and to others:

https://discuss.python.org/t/pep-810-explicit-lazy-imports/1...

The specific part of the PEP:

https://github.com/python/peps/pull/4628/files#diff-ca011267...

Post reply on HN