Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python for Humans

python-for-humans.heroku.com

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Python for Humans

#31

Post by socratic »

This presentation brings up a tangential point that has always confused me: how error-prone is starting a subprocess, really?

I agree with the author's goals of making common tasks easier and more obvious. urllib2 is an easy target, as it was added to the standard library over a decade ago, long before REST was something people talked about. The best tools for packaging, versioning, and testing have always been a bit ambiguous in any language, including Python.

However, the author points out something that has always bothered me about Python: it is way harder to start a subprocess with an external command in Python than almost any other language. This has been true whether using sys or os or even subprocess, which is quite recent.

I always felt that this had something to do with the constant warnings in the documentation about how a pipe between the subprocess and the Python process might fill and cause the subprocess to block. Or how running the program through shell rather than exec or something might cause some sort of security issue. Are these real issues that other languages ignore in the name of user convenience, or has Python just never been able to make the right API (as the author seems to argue)?

Re: Python for Humans

#32

Post by bobbyi »

loevborg wrote: post #30

I was intrigued by the author's library "envoy", which is intended to provide a more intuitive interface to running processes from python. ( https://github.com/kennethreitz/envoy ) The back story is that the older APIs that Python comes with -- os.popen and os.system -- are deprecated. Programmers are urged to use the "subprocess" module instead. Although this doesn't have the problems of the original functions, it h…

> it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess.

  oputput = subprocess.check_output(my_command)

Re: Python for Humans

#33

Post by tbatterii »

rmc wrote: post #11

This makes a lot of good points, but some bad ones. Esp. the "installing python" one. Just use your package manager to install all the versions you need. And for "Packaging and Dependencies", just use pip.

What's the package manager for windows?

Mostly I agree with you, but installing python on windows is a PITA compared to *nix.

PIP works until you need something that requires compiling extensions which is also a PITA on windows.

IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time.

Regardless, my point is, though I think we all wish it was as simple as using your package manager and pip, it simply isn't that way for everyone.

And it probably never will be, but we can always make it simpler. :)

Re: Python for Humans

#35

Post by bretthoerner »

socratic wrote: post #31

This presentation brings up a tangential point that has always confused me: how error-prone is starting a subprocess, really? I agree with the author's goals of making common tasks easier and more obvious. urllib2 is an easy target, as it was added to the standard library over a decade ago, long before REST was something people talked about. The best tools for packaging, versioning, and testing have always been a bit…

It's funny you mention that. The author/speaker wrote a "Subprocesses for humans" module, too: https://github.com/kennethreitz/envoy

There's no fundamental problem that's stopped Python from doing this before. For some reason, all of the ways to spawn a subprocess in Python have tried to map almost directly to the underlying C API... which is pretty awful.

Re: Python for Humans

#36

Post by huyegn »

socratic wrote: post #31

This presentation brings up a tangential point that has always confused me: how error-prone is starting a subprocess, really? I agree with the author's goals of making common tasks easier and more obvious. urllib2 is an easy target, as it was added to the standard library over a decade ago, long before REST was something people talked about. The best tools for packaging, versioning, and testing have always been a bit…

To get around the problem of child subprocesses spewing out too much output and blocking the parent process, one can provide an open file handle to the stdout/stderr arguments of the Popen call. I've ran into this many times and this solution has reliably worked for me every time. This could be documented better in the Python docs.

For quick tasks and scripts, I've found subprocess.check_call, and subprocess.check_output with shell=True are great tools for spawning subprocesses and quickly grabbing output. They're pretty straightforward to use.

Re: Python for Humans

#37

Post by j_baker »

rmc wrote: post #11

This makes a lot of good points, but some bad ones. Esp. the "installing python" one. Just use your package manager to install all the versions you need. And for "Packaging and Dependencies", just use pip.

What if your package manager doesn't support the version of Python you're targeting? The most common place this happens is if you have some old RHEL 5 boxen that haven't been upgraded (and who that uses RHEL doesn't?). Or suppose you use python 2.7, which isn't supported by very many (any?) "stable" linuces (aka Debian stable, RHEL, Ubuntu LTR, etc).

And installing packages into the system python (if that's what you're suggesting) is the path to madness. It's much better to use virtualenvs you can throw away at will. All in all, it's usually best just to leave the system Python alone to avoid causing problems with any other packages that may depend on it being in a consistent state.

Re: Python for Humans

#38

Post by rmc »

tbatterii wrote: post #33
rmc wrote: post #11

This makes a lot of good points, but some bad ones. Esp. the "installing python" one. Just use your package manager to install all the versions you need. And for "Packaging and Dependencies", just use pip.

What's the package manager for windows? Mostly I agree with you, but installing python on windows is a PITA compared to *nix. PIP works until you need something that requires compiling extensions which is also a PITA on windows. IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time. Regardless, my point is, though…

I would not use Windows for servers.

Re: Python for Humans

#39

Post by j_baker »

rmc wrote: post #38
tbatterii wrote: post #33

Earlier quoted context omitted.

What's the package manager for windows? Mostly I agree with you, but installing python on windows is a PITA compared to *nix. PIP works until you need something that requires compiling extensions which is also a PITA on windows. IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time. Regardless, my point is, though…

I would not use Windows for servers.

Neither would I. But some people are stuck in jobs where they have to support Windows. If that's never been the case for you, consider yourself lucky.

Re: Python for Humans

#40

Post by tbatterii »

rmc wrote: post #38
tbatterii wrote: post #33

Earlier quoted context omitted.

What's the package manager for windows? Mostly I agree with you, but installing python on windows is a PITA compared to *nix. PIP works until you need something that requires compiling extensions which is also a PITA on windows. IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time. Regardless, my point is, though…

I would not use Windows for servers.

neither would I. but I'm not pretending they don't exist to prove a point :)

Whether you or I prefer it or not, there are lots of people who either by choice or not develop on windows and in some cases deploy to windows servers.

And for those people, things aren't as simple as "use the package manager and pip" no matter how much anyone would like it to be.

reality sucks.

Post reply on HN