Live data from Hacker News

Fixing the Python subprocess interface

amoffat.github.com

51–60 of 65 posts

Re: Fixing the Python subprocess interface

#53
post #50
post #8

It's odd that the import mechanism is abused here to make objects "out of thin air". The fact that "from pbs import ffmpeg" works only if ffmpeg is actually on the path is somewhat surprising. I think the more comfortable (and Pythonic?) way to do this would be to explicitly create these command objects: >>> import pbs >>> ffmpeg = pbs.Command('ffmpeg') # or '/usr/bin/ffmpeg', perhaps >>> result = ffmpeg(...) [Edit:…

Hi! Author here, there are a few ways to use it, including your suggestion (did you make your suggestion up or were you pulling from the docs?): # magical, designed only for single shell scripts from pbs import * ffmpeg() # less magical from pbs import ffmpeg ffmpeg() # or import pbs pbs.ffmpeg() # no magic import pbs ffmpeg = pbs.Command(pbs.which("ffmpeg")) # command takes full path ffmpeg() I tried to cover the ma…

This looks really cool. It was not clear to me on the first reading of the document that it could be invoked these ways. The pbs.Command() example is way down in the "weird filenames" part. I just did a quick change to the README and sent you a pull request.

Re: Fixing the Python subprocess interface

#54
post #50
post #8

It's odd that the import mechanism is abused here to make objects "out of thin air". The fact that "from pbs import ffmpeg" works only if ffmpeg is actually on the path is somewhat surprising. I think the more comfortable (and Pythonic?) way to do this would be to explicitly create these command objects: >>> import pbs >>> ffmpeg = pbs.Command('ffmpeg') # or '/usr/bin/ffmpeg', perhaps >>> result = ffmpeg(...) [Edit:…

Hi! Author here, there are a few ways to use it, including your suggestion (did you make your suggestion up or were you pulling from the docs?): # magical, designed only for single shell scripts from pbs import * ffmpeg() # less magical from pbs import ffmpeg ffmpeg() # or import pbs pbs.ffmpeg() # no magic import pbs ffmpeg = pbs.Command(pbs.which("ffmpeg")) # command takes full path ffmpeg() I tried to cover the ma…

Wow -- no, I entirely missed this (and just happened to end up with the same name). Thanks for clarifying.

Re: Fixing the Python subprocess interface

#55
Reminds me of something I saw not too long ago...

https://github.com/JulienPalard/Pipe

It would be really cool (though admittedly less Pythonic) to combine the infix notation provided by the Pipe library to allow more shell-like function chaining.

Instead of this...

  print wc(ls("/etc", "-1"), "-l")
You would have this...

  print ls("/etc", "-1") | wc("-l")

Re: Fixing the Python subprocess interface

#56

FWIW, re-replaying the stack trace to figure out what was imported and re-implementing it is a horrible idea. There are much better ways to do this type of import voodoo, specifically the import hooks that Python ships with. Here's an example of their use inside a small side project of mine: https://github.com/tswicegood/maxixe/blob/master/maxixe/__in... All that said, this is horribly un-Pythonic. A much better rout…

envoy looks a lot more verbose than this though . . . if you're really trying to replace shell scripting, having envoy.run(foo) on every line is going to get annoying. Also, it's not clear on what basis you make this assertion: > FWIW, re-replaying the stack trace to figure out what was imported and re-implementing it is a horrible idea. Does it not work?

"Does it… work?" is not the standard by which good Python code is measured. Django went through this process years ago. It's convenient to be able to call `from myapp.models import *` and have an `Articles` model magically added to your module even though you never defined. They realized years ago that writing clever code for the sake of being clever was a bad idea. How are you going to get someone else to be able to maintain it? Not only do they have to know the logic of what you're doing, they have to know how you hacked things together.

Line 17 of PEP-20 sums up my thoughts on this code as more than an intellectual exercise.

> If the implementation is hard to explain, it's a bad idea.

Regarding using Python as a replacement for shell scripting: if you have to rewire the language to do what you want, why are you using a different tool for the job?

Re: Fixing the Python subprocess interface

#57
post #55

Reminds me of something I saw not too long ago... https://github.com/JulienPalard/Pipe It would be really cool (though admittedly less Pythonic) to combine the infix notation provided by the Pipe library to allow more shell-like function chaining. Instead of this... print wc(ls("/etc", "-1"), "-l") You would have this... print ls("/etc", "-1") | wc("-l")

There's a few ideas floating around here https://github.com/amoffat/pbs/issues/6 on how to implement it, but nothing looks really feasible. If you have any insights, I welcome them :)

Re: Fixing the Python subprocess interface

#58
post #51

This looks great, but I really don't like the fact that I can't fire up a Python shell and try it out interactively. Having to run pbs.py itself to get a different kind of shell is uncomfortable.

This is fixed on master as of version 0.4, just fyi. It has some limitations (no star import) but otherwise works as expected.

Re: Fixing the Python subprocess interface

#59

FWIW, re-replaying the stack trace to figure out what was imported and re-implementing it is a horrible idea. There are much better ways to do this type of import voodoo, specifically the import hooks that Python ships with. Here's an example of their use inside a small side project of mine: https://github.com/tswicegood/maxixe/blob/master/maxixe/__in... All that said, this is horribly un-Pythonic. A much better rout…

Just to clarify. I applaud this type of development for people trying to learn various parts of Python like playing with the stack, but the idea of this being used in the wild scares me a bit.

It's like operator overloading for novel syntax in C++. It worked all right for iostream, but if you've ever seen boost::spirit you can see that even very smart people can make very weird things happen by trying to kludge features into new syntax.

Re: Fixing the Python subprocess interface

#60

Earlier quoted context omitted.

envoy looks a lot more verbose than this though . . . if you're really trying to replace shell scripting, having envoy.run(foo) on every line is going to get annoying. Also, it's not clear on what basis you make this assertion: > FWIW, re-replaying the stack trace to figure out what was imported and re-implementing it is a horrible idea. Does it not work?

"Does it… work?" is not the standard by which good Python code is measured. Django went through this process years ago. It's convenient to be able to call `from myapp.models import *` and have an `Articles` model magically added to your module even though you never defined. They realized years ago that writing clever code for the sake of being clever was a bad idea. How are you going to get someone else to be able to…

> If the implementation is hard to explain, it's a bad idea.

I don't think this principle is universal in software, and I don't think you need to embrace it as a prerequisite for writing or distributing Python code. My personal preference would be for a statement more like this, "If the implementation is more complicated than it needs to be to do what you want, it's probably a bad idea."

> Regarding using Python as a replacement for shell scripting: if you have to rewire the language to do what you want, why are you using a different tool for the job?

Well that's easy to answer. It's cleaner and more readable than bash, has some nice features that shell scripts lack, can be used to call into python libraries, allows a single unified codebase if you're already writing python code . . . I could go on.

Post reply on HN