Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Python Standard Library - Where Modules Go To Die

leancrew.com

Post reply on HN
Print view View original

31–40 of 75 posts

Re: The Python Standard Library - Where Modules Go To Die

#31

Post by chaostheory »

aslewofmice wrote: post #14
norswap wrote: post #13

The thing with python standard library is that it is crapily documented imho. I often can't make heads or tails of it, while I have a much simpler time with any other language (you name it: Java, Ruby, PHP, C, Scala, Lisp, ...).

I felt that way about some of the standard documentation but was relieved to find: http://www.doughellmann.com/PyMOTW/ I also highly recommend checking out Doug Hellman's book 'The Python Standard Library by Example'. He presents every (or almost every) standard library module with simple explanations and plenty of examples.

Nice post. This is way better than the standard docs. Having lots of examples really helps.

Re: The Python Standard Library - Where Modules Go To Die

#32

Post by jvdongen »

jordanb wrote: post #26
JoachimSchipper wrote: post #15

Earlier quoted context omitted.

It is stable, but there are some really nasty warts. To pick just one that bites Python programmers all the time : by default, the ssl library does not validate the server certificate at all. Not validating the certificate makes SSL/TLS almost useless. But this is still the default (see http://docs.python.org/dev/library/ssl.html#socket-creation , "CERT_NONE"), because the standard library is "stable".

On the contrary, the vast majority of the time, when one is using SSL, they're using it because they want encryption, rather than identification. The certificate system surrounding SSL is a complete mess. It does virtually nothing other than trigger false positives for people who who haven't paid the appropriate "security partner." The very rare person who is actually using SSL for identification rather than just to…

I now see your 'PS' (perhaps added while I was adding my comment?) - however I cannot follow the argument you make.

IF you assume that MITM-attacks are rare, you probably also assume that traffic snooping is rare (which is after all a form of a MITM-attack). If that's the case, why use encrypted communication channels at all?

Security is never perfect - it always is about adding layer upon layer to make the bar high enough that the remaining number of adversaries becomes more manageable.

Spoofing a site that is not using SSL is trivial. Using SSL with public CA signed certificates significantly raises the bar. Not to the 'perfect' level, but enough to make a real difference. Not checking the server certificate throws you back to the 'trivial' level.

Re: The Python Standard Library - Where Modules Go To Die

#33

Post by jbri »

jordanb wrote: post #26
JoachimSchipper wrote: post #15

Earlier quoted context omitted.

It is stable, but there are some really nasty warts. To pick just one that bites Python programmers all the time : by default, the ssl library does not validate the server certificate at all. Not validating the certificate makes SSL/TLS almost useless. But this is still the default (see http://docs.python.org/dev/library/ssl.html#socket-creation , "CERT_NONE"), because the standard library is "stable".

On the contrary, the vast majority of the time, when one is using SSL, they're using it because they want encryption, rather than identification. The certificate system surrounding SSL is a complete mess. It does virtually nothing other than trigger false positives for people who who haven't paid the appropriate "security partner." The very rare person who is actually using SSL for identification rather than just to…

If MitM attacks are so rare, why bother encrypting your traffic in the first place? Packet-snooping attacks are also "extremely rare" by most metrics, so why protect against one but not the other?

Either go all the way on security, or be obvious about not having any. Appearing secure when in actuality you're not is the worst option.

Re: The Python Standard Library - Where Modules Go To Die

#34

Post by TazeTSchnitzel »

norswap wrote: post #13

The thing with python standard library is that it is crapily documented imho. I often can't make heads or tails of it, while I have a much simpler time with any other language (you name it: Java, Ruby, PHP, C, Scala, Lisp, ...).

The Python standard library has great documentation.

Except for some of the "batteries included" stuff. urllib2? Ouch.

Re: The Python Standard Library - Where Modules Go To Die

#35

Post by IgorPartola »

Agreed with the OP. The following is a shameless plug:

Python's ConfigParser module is a pain to use. It provides no validation, only supports a limited number of types of data you can retrieve, etc. Similarly, getopt vs optparse vs argparse is a mess. getopt is universal: not only is it going to be in all versions of Python, but it is also the same library available in virtually every other language. The problem with it is that it is not declarative, so you will typically see a giant if/elif statement that goes with it. argparse/optparse are better, but aren't universal even between versions of Python, though argparse has been backported and is available via pypi.

To unify all this into one convenient module, I ended up writing http://ipartola.github.com/groper/. groper lets you specify your parameters declaratively, and if you specify defaults, use them right away without having to create/modify a config file. It automatically figures out the priority of arguments: cmd > config > defaults. It also has some niceties such as the ability to automatically generate usage strings, give the user intelligent error messages, generate sample config files, etc.

Re: The Python Standard Library - Where Modules Go To Die

#36

Post by nkoren »

espeed wrote: post #29
nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my environment is set up for. Create a virtual environment for the Python version you want to use, and install the package with pip (pip will install all the dependencies in your project's local environment)... $ mkdir myproj $ cd myproj $ vir…

Yep, I learned this trick on my first go-around. The problem has occurred when I've tried to run scripts with dependencies which somehow are only available in mutually incompatible versions of Python. I'm not sure how this is possible (it's baffling -- it shouldn't be possible), but it's happened three out of four times that I've tried to use a Python script of any real consequence. Usually after half a day of futilely trying to find an environment which will actually accommodate all dependencies, I end up having to port everything to whatever version of Python appears to be the most common denominator. In all fairness to Python, this is relatively easy to do (except in one case, when I had to hire a Python-expert friend to do it), but it still means that what should've been a five-minute affair (less the dependency hell) turns into a full-day affair.

Like I say, I've probably just been unlucky. But at this point it's given me a pretty serious aversion to Python. Will probably have to get over that someday, I suppose.

Re: The Python Standard Library - Where Modules Go To Die

#37

Post by eliben »

Yes, for a language as widely deployed and used as Python, retaining backwards compatibility and stability is more important than adding new and shiny tools to the stdlib at a faster pace. Users rely on the fact that a module in stdlib will remain there and will remain stable for a long time. More modules means more maintainers, and Python is an open-source project developed by volunteers. It's that simple.

I'm not sure what the solution this article proposes is. The tradeoff between "coolness" and "stability" is inherently difficult, and I'm sure Python is not the only language "suffering" from it.

After all, it's quite easy to install a new Python module, and not much harder to distribute it with your application (for web apps it's even easier), so what is the problem?

Re: The Python Standard Library - Where Modules Go To Die

#38

Post by randallsquared »

jbri wrote: post #33
jordanb wrote: post #26

Earlier quoted context omitted.

On the contrary, the vast majority of the time, when one is using SSL, they're using it because they want encryption, rather than identification. The certificate system surrounding SSL is a complete mess. It does virtually nothing other than trigger false positives for people who who haven't paid the appropriate "security partner." The very rare person who is actually using SSL for identification rather than just to…

If MitM attacks are so rare, why bother encrypting your traffic in the first place? Packet-snooping attacks are also "extremely rare" by most metrics, so why protect against one but not the other? Either go all the way on security, or be obvious about not having any. Appearing secure when in actuality you're not is the worst option.

Packet-snooping attacks are also "extremely rare" by most metrics [...]

Really? NSA boxes in AT&T (and presumably other) switching stations suggest that for US traffic it's extremely common.

Re: The Python Standard Library - Where Modules Go To Die

#39

Post by anthonyb »

zeeg wrote: post #16
true_religion wrote: post #11

Earlier quoted context omitted.

How is it not well defined in Python?

Well it would work just fine, but you'd always end up requiring the external dependency even if you didn't need to. For example, let's say there was a new urllib released (its still called urllib). It's now version 2.0, but the stdlib version is 1.0. If your package said "I need urllib==1.0", it would have know way of understanding that the version was already included within the standard library. That said, it would…

> it would have know way of understanding that the version was already included within the standard library

Other than by introspecting which packages are installed, that is. Most of them will have a VERSION, __version__ or _version attribute which tells you.

Re: The Python Standard Library - Where Modules Go To Die

#40

Post by anthonyb »

nkoren wrote: post #19

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my envi…

I hate PHP's standard library - I find it to be weird and inconsistent, arguments are in a random order, etc. and Python to be not so bad. I suspect this is because you have a lot of PHP experience, and I have a lot of Python experience :)

Virtualenv and PIP go a long way towards fixing these problems in Python, similar to how PEAR and CPAN work with PHP and Perl.

Post reply on HN