Say “no” to import side‐effects in Python • Hacker News Board

Re: Say “no” to import side‐effects in Python

#41

Post by marcosdumay » Sat, Apr 05, 2014, 2:17 PM UTC

einhverfr wrote: post #29

Mithaldu wrote: post #24

Earlier quoted context omitted.

In Perl the act of simply importing that patch_all function would be a side-effect, as it's implemented as code in pure perl that changes the caller's symbol table. I suspect it would be fair to call your import up there also a side-effect, but that it slips the mind since it's implemented in the python core. :)

Indeed. Also there are times (Moose for example) where you want to do more than import a function as written. For example Moo and Moose create a custom function "has" at import time that is them attached to the caller's namespace. The reason for this is that you want to fully attach the function in the space as a native method and this requires some closures to make work sanely. In short you want: package foo; use Mo…

Technically, you can do this with Python, but you'll have to use some dark magic to get the importing module namespace, in order to manipulate it. It'll not be pretty.

Instead, in normal Python it is the importing module that must decide who has access to its own namespace, and who must live within a separated one. It has a completely different set of costs and benefits. (I sometimes wonder if Guido used Perl as a counterexample when creating Python - its principles are almost completely opposite.)

Re: Say “no” to import side‐effects in Python

#42

Post by kzrdude » Sat, Apr 05, 2014, 2:17 PM UTC

chrismorgan wrote: post #27

ghfdaghkj wrote: post #25

Earlier quoted context omitted.

I haven't used GTK in a long time, but when I was using it, I was under the impression that pygtk was being deprecated, and everyone should be using GObject-Introspection instead.

I dunno—I was making assumptions, not being very familiar with the GTK scene in Python.

Not me either anymore. I tried to port to introspection + Gtk3 but ran into problems with widget subclasses and no documentation to resolve it with.

Re: Say “no” to import side‐effects in Python

#43

Post by marcosdumay » Sat, Apr 05, 2014, 2:19 PM UTC

aptwebapps wrote: post #5

This in no way invalidates the point of the post, but I can't imagine going back to installing everything globally instead of using virtualenv. If you do that, at least you won't have every package you every looked at in your path.

I just can't get to use some modules within virtualenv. They seem to create more problems than using systemwide.

Re: Say “no” to import side‐effects in Python

#44

Post by bjt » Sat, Apr 05, 2014, 2:23 PM UTC

atilaneves wrote: post #28

I have the unfortunate "luck" to be using a Python library at work that _loves_ to use import side-effects. Importing it like God intended makes it parse command-line arguments and fail if it doesn't like what was passed in. And that's just the start. Nearly every __init__.py has code in it, including class definitions. I have no idea why.

In case a non-Python programmer is reading this and misunderstood your comment, I'll point out that having code in __init__.py and having import-time side effects are completely unrelated.

When I put any code in __init__.py, it's usually just to make the import path shorter for the programmer using my library. So instead of this:

from mypackage.models import Foo

...users can do this:

from mypackage import Foo

And all it takes to support that is putting this in mypackage/__init__.py:

from mypackage.models import Foo

A quick Googling of "Should I put code in __init__.py" shows a lot of people doing that same pattern. It doesn't show a consensus of people saying that more substantial code in __init__.py is an anti-pattern.

Re: Say “no” to import side‐effects in Python

#45

Post by Argorak » Sat, Apr 05, 2014, 2:23 PM UTC

mercurial wrote: post #33

TazeTSchnitzel wrote: post #30

This is one reason why I prefer Haskell ;)

I think most statically typed languages don't care for this kind of shenanigans.

Java for example runs arbitrary code when a class is loaded (static {} blocks).

Windows has DllMain, which lets you run code at load of a shared lib and similar things are possible on Linux, using the -init linker flag and friends.

Finally, Haskell has features that allow you to load, compile and run Haskell (through the GHCMonad, if installed), which allows precisely for those shenanigans.

All those lack the habit that the OP is describing (and which broke for him): the use runtime introspection during development time (possibly in a REPL), but that's another pair of shoes.

Re: Say “no” to import side‐effects in Python

#46

Post by marcosdumay » Sat, Apr 05, 2014, 2:27 PM UTC

Hello71 wrote: post #22

$ echo 3 > /proc/sys/vm/drop_caches $ time python2 -c 'help("modules")' >/dev/null /usr/lib64/python2.7/site-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion 'g_type_from_name (name) == 0' failed import gobject._gobject python2 -c 'help("modules")' > /dev/null 1.58s user 0.23s system 69% cpu 2.626 total $ time python3 -c 'help("modules")' > /dev/null python3 -c 'help("modules")' > /d…

Wow! My Pthon2 help('modules') opens a window titled "Hello from wxPython"!

Python3 behaves well, and both succeed...

Re: Say “no” to import side‐effects in Python

#47

Post by pekk » Sat, Apr 05, 2014, 3:10 PM UTC

marcosdumay wrote: post #43

aptwebapps wrote: post #5

This in no way invalidates the point of the post, but I can't imagine going back to installing everything globally instead of using virtualenv. If you do that, at least you won't have every package you every looked at in your path.

I just can't get to use some modules within virtualenv. They seem to create more problems than using systemwide.

Which ones? You can get some help. Learning virtualenv is a smart long-term move

Re: Say “no” to import side‐effects in Python

#48

Post by pekk » Sat, Apr 05, 2014, 3:12 PM UTC

jzwinck wrote: post #3

Of course you should do this in any language. I once used a Ruby library that, when loaded, would try to connect to a database on a remote machine. Programs which required this library would take several seconds to display their --help output. Because of that and similar incidents, I've learned to import argparse up front but nothing else unless necessary. Once argument parsing is done, then importing other modules b…

It's nice to have the imports up front. Are you sure you can't give this special treatment to the modules that need it rather than doing it to literally all your dependencies?

Re: Say “no” to import side‐effects in Python

#49

Post by gbog » Sat, Apr 05, 2014, 3:16 PM UTC

atilaneves wrote: post #38

mercurial wrote: post #32

Earlier quoted context omitted.

It's like programmers who find out about metaprogramming. Usually they grow out of it.

I haven't grown out of metaprogramming at all. I just try and use it only when it's the best option. Usually because it means reducing duplication. I liked metaprogramming to an extent in C++. I _love_ it in D.

Replace "best option" with "only practical option" and you're good to go.

Re: Say “no” to import side‐effects in Python

#50

Post by gbog » Sat, Apr 05, 2014, 3:21 PM UTC

For those not in the knowing, avoiding import side effect means never having top level code in any python module except class, functions and constant definitions, other imports and the occasional if name = main. And constants must be built-in types.