Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Python Standard Library - Where Modules Go To Die

leancrew.com

Post reply on HN
Print view View original

21–30 of 75 posts

Re: The Python Standard Library - Where Modules Go To Die

#21

Post by heretohelp »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

I think you've really just been unlucky.

I've had the same experience before, but chiefly with Ruby.

Interestingly, my problems with this in both Python and Ruby have evaporated once I got in the habit of using virtualenv/rb-env/rvm for my development environment.

Python is about the cleanest/nicest experience I have in any language, for the record. Only language that comes close is Clojure.

Leiningen is...legendary.

Re: The Python Standard Library - Where Modules Go To Die

#22

Post by lloeki »

chimeracoder wrote: post #7

> An overstatement, certainly, but with more than a germ a truth. Once a library is enshrined in the standard set, it can’t change radically because too many programs rely on it—and its bugs, idiosyncrasies, and complications—remaining stable. That's a problem inherent in the standardization process, though - it's all but contradictory to have something be both 'standard' and 'continuously improving'. Once something…

Things come and go, and we have seen a number of deprecations in python already. urllib predates urllib2, while subprocess deprecates a number of things itself that really came from C. getopt was a port of the eponymous C library, which optparse meant to replace, which was itself deprecated on favor of argparse.

I would really not be surprised to see envoy, requests and so on come up in the standard lib at some point.

Re: The Python Standard Library - Where Modules Go To Die

#23

Post by lloeki »

100k wrote: post #12
lars512 wrote: post #4

Earlier quoted context omitted.

I find the opposite. Installing a useful module for a once-off task is a no brainer, since I won't have to worry about whether I'm introducing some long-term dependency I'll have to maintain. The real issue is in discovering that a better option exists in the first place.

And then you have to deploy it to the server or someone else's computer and it sucks.

pip freeze works even without virtualenv. Expunge things you know don't matter, and send to third party/deploy, which then just has to run pip install -r requirements.txt.

Re: The Python Standard Library - Where Modules Go To Die

#24

Post by [deleted] »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

[deleted]

Re: The Python Standard Library - Where Modules Go To Die

#25

Post by chaostheory »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

I think the problem is because when you used it the Python community probably still hadn't decided on a one true dependency system like Gems / rvm for Ruby, and Maven for Java. It's been about four years since I've heavily used Python, but hopefully this has been fixed by now.

I don't have this problem on either Ruby or Java.

Re: The Python Standard Library - Where Modules Go To Die

#26

Post by jordanb »

JoachimSchipper wrote: post #15
atdt wrote: post #9

The fact that the standard library is well-maintained, carefully debugged, and backward-compatible is a far stronger indicator of Python's awesomeness than the existence of shiny, new libraries. Hackers naturally gravitate toward high-visibility projects with brave horizons and bold scopes. By contrast, it is incredibly hard to find the motivation to update, for the umpteenth time, a warty API -- and that's precisely…

It is stable, but there are some really nasty warts. To pick just one that bites Python programmers all the time : by default, the ssl library does not validate the server certificate at all. Not validating the certificate makes SSL/TLS almost useless. But this is still the default (see http://docs.python.org/dev/library/ssl.html#socket-creation , "CERT_NONE"), because the standard library is "stable".

On the contrary, the vast majority of the time, when one is using SSL, they're using it because they want encryption, rather than identification.

The certificate system surrounding SSL is a complete mess. It does virtually nothing other than trigger false positives for people who who haven't paid the appropriate "security partner."

The very rare person who is actually using SSL for identification rather than just to establish an encrypted TCP connection, and therefore cares about certificates, can change the default.

PS: I know the standard response to this, that encryption without identification is useless, because without identification your counter-party might be Eve. In reality, in the real world, that doesn't happen. MITM attacks are extremely rare. And the real Eves on the net (phishers) can easily obtain signed certificates that will fool pretty much any end user.

Re: The Python Standard Library - Where Modules Go To Die

#27

Post by fuzzix »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

Most frustrations I've had with Perl/Ruby/Python have been down to inconsistent, incomplete or out-of-date packaging by the software distribution.

I've found I can mitigate most of this by using perlbrew/cpanminus, pythonbrew/pip and rvm/gem.

This adds complications when it comes to deployment - you've added additional maintenance dependencies to your servers if you don't stick with what the package manager provides... Nothing insurmountable, but keeping an eye on updates (would usually use email to notify on dated versions) and maybe having your CI system always use the latest versions of everything are possible methods. I'd like to hear better ideas, to be honest.

My background is both sysadmin and development, I think there should be no reason we can't make this all simpler for everyone.

Re: The Python Standard Library - Where Modules Go To Die

#28

Post by zokier »

RegEx wrote: post #10
atdt wrote: post #5

Earlier quoted context omitted.

> the Python module install pain for a one-off task. "pip install requests" ?

Unfortunately, it seems the official documentation on installing Python modules[0] makes absolutely no mention of pip or even easy_install. Seems like something that should be there, right? [0]: http://docs.python.org/install/

I guess that is because pip is not part of the official Python. Maybe best described as a front-end to distutils (which is what your link documents and is part of official Python).

edit: Python docs front-page[1] also notes following:

A new documentation project, which will be merged into the Python documentation soon, covers creating, installing and distributing Python packages: http://guide.python-distribute.org/

[1] http://www.python.org/doc/

So I guess that they have acknowledged that the docs are suboptimal currently in this part.

Re: The Python Standard Library - Where Modules Go To Die

#29

Post by espeed »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my environment is set up for.

Create a virtual environment for the Python version you want to use, and install the package with pip (pip will install all the dependencies in your project's local environment)...

  $ mkdir myproj
  $ cd myproj
  $ virtualenv --python=python2.7 env
  $ source env/bin/activate
  (env)$ pip install somepackage

Re: The Python Standard Library - Where Modules Go To Die

#30

Post by jvdongen »

jordanb wrote: post #26
JoachimSchipper wrote: post #15

Earlier quoted context omitted.

It is stable, but there are some really nasty warts. To pick just one that bites Python programmers all the time : by default, the ssl library does not validate the server certificate at all. Not validating the certificate makes SSL/TLS almost useless. But this is still the default (see http://docs.python.org/dev/library/ssl.html#socket-creation , "CERT_NONE"), because the standard library is "stable".

On the contrary, the vast majority of the time, when one is using SSL, they're using it because they want encryption, rather than identification. The certificate system surrounding SSL is a complete mess. It does virtually nothing other than trigger false positives for people who who haven't paid the appropriate "security partner." The very rare person who is actually using SSL for identification rather than just to…

Not to pick on you per se, but this reasoning is often seen in relation to SSL (or encryption in general) and is dangerously wrong.

Encryption without identification and authentication of your communication partners is useless. You may very well end up with a very secure link with the wrong communication partner (google 'man-in-the-middle-attack').

I agree that the (public) CA system is a mess, however especially with machine-to-machine communication it is very easy to generate, sign and use your own certificates. And contrary to popular belief, self-signed certificates are not any less secure than public CA signed ones. Both have their own use-case though.

If someone cares I'll be happy to explain the above points in more detail.

Post reply on HN