Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CLI tools hidden in the Python standard library

til.simonwillison.net

Post reply on HN
Print view View original

11–20 of 160 posts

Re: CLI tools hidden in the Python standard library

#11

Post by BiteCode_dev »

lizard wrote: post #2

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path.

This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path.

It is my favorite way of running my projects.

Unfortunalty it means you can't use "-m pdb", and that's a big loss.

Re: CLI tools hidden in the Python standard library

#13

Post by polyrand »

Python 3.12 will include a SQLite CLI/REPL in the standard library too[0][1]. This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI.

[0]: https://github.com/python/cpython/blob/3fb7c608e5764559a718c...

[1]: https://docs.python.org/3.12/library/sqlite3.html#command-li...

Re: CLI tools hidden in the Python standard library

#14

Post by formerly_proven »

mg wrote: post #10

There is one problem with those: Security. Using modules (even if they are in the standard library) on the command line lets malicious code in the current dir take over your machine: https://twitter.com/marekgibney/status/1598706464583028736

iirc this is one of the things earmarked for a hypothetical Python 4, making -P the default. It's also one of the many relatively well-known (security) issues in Python that don't get addressed for a surprising amount of time. Others in the same vein would be stuff like stderr being block-buffered when not using a TTY, no randomized hashes for the longest time, loading DLLs preferably from the working directory, std* using 7-bit ASCII in a number of circumstances and many more.

Re: CLI tools hidden in the Python standard library

#15

Post by scruple »

BiteCode_dev wrote: post #11
lizard wrote: post #2

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

I also develop and run projects this way. I really, really enjoy it. It's a very pleasant experience, on both the development side and execution side.

I'm relatively new to Python (used it for ~1 year in 2007/2008, again briefly in 2014 -- which is when I believe I picked this module trick up -- and then didn't touch it again until March of this year). It's made an impression on my team and we're all having a good time developing code this way. I do wonder, though, what other shortcomings might exist with this approach.

Re: CLI tools hidden in the Python standard library

#16

Post by andreareina »

BiteCode_dev wrote: post #11
lizard wrote: post #2

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

I've taken to adding a --pdb option to my scripts.

Re: CLI tools hidden in the Python standard library

#17

Post by chmaynard »

polyrand wrote: post #13

Python 3.12 will include a SQLite CLI/REPL in the standard library too[0][1]. This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI. [0]: https://github.com/python/cpython/blob/3fb7c608e5764559a718c... [1]: https://docs.python.org/3.12/library/sqlite3.html#command-li...

> This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI.

Not sure what you mean -- sqlite3 is the SQLite CLI.

Re: CLI tools hidden in the Python standard library

#18

Post by sharikous »

BiteCode_dev wrote: post #11
lizard wrote: post #2

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

You can use pdb

Just

python -m pdb -m module

Re: CLI tools hidden in the Python standard library

#19

Post by OJFord »

dmurray wrote: post #6
Noumenon72 wrote: post #5

python -m json.tool seems neat. I am always dumping JSON in single log lines and then expanding it to diff it (currently using Cmd+Opt+L in PyCharm).

I tend to use "jq .". An extra tool, but you don't really have to learn it, and if you're "always dumping JSON" and also use the command line a lot, you probably want to have it around anyway.

You don't need the `.` fwiw, just `produce-compact-json | jq` will do.

Re: CLI tools hidden in the Python standard library

#20

Post by TeMPOraL »

formerly_proven wrote: post #14
mg wrote: post #10

There is one problem with those: Security. Using modules (even if they are in the standard library) on the command line lets malicious code in the current dir take over your machine: https://twitter.com/marekgibney/status/1598706464583028736

iirc this is one of the things earmarked for a hypothetical Python 4, making -P the default. It's also one of the many relatively well-known (security) issues in Python that don't get addressed for a surprising amount of time. Others in the same vein would be stuff like stderr being block-buffered when not using a TTY, no randomized hashes for the longest time, loading DLLs preferably from the working directory, std*…

> loading DLLs preferably from the working directory

That is a feature, though, isn't it?

Post reply on HN