Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CLI tools hidden in the Python standard library

til.simonwillison.net

Post reply on HN
Print view View original

51–60 of 160 posts

Re: CLI tools hidden in the Python standard library

#51

Post by sharikous »

kzrdude wrote: post #38
BiteCode_dev wrote: post #27

Earlier quoted context omitted.

Damn, 15 years of python and I learn you can use -m twice. I've never even tried, didn't occur to me it would be supported. EDIT: it support other options as well, like -c. That deserves an alias: debug_module() { if python -c "import ipdb" &>/dev/null; then python -m ipdb -c c -m "$@" else python -m pdb -c c -m "$@" fi }

There's no magic, only layers. `python -m pdb ` runs pdb with the rest of the arguments. pdb handles the second `-m`. If you have a fancy IDE feature, open a new python file, type "import pdb", use go to definition on pdb to jump to that file in the standard library, and read its main function - it handles -m explicitly :)

Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm?

Multiline statements are not accepted, nor things like if/for

Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL

Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab are very used to a trial and error programming style in a console and bare python does not address this need

Re: CLI tools hidden in the Python standard library

#52

Post by quietbritishjim »

BiteCode_dev wrote: post #11
lizard wrote: post #2

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

> which means suddenly relative imports works

Not just relative imports but also (properly formed) absolute imports. For example, if you have a directory my_pkg/ with files mod1.py and mod2.py, then

    # In my_pkg/mod2.py
    import my_pkg.mod1
will work if you run `python -m my_pkg.mod2` but will fail if you run `python my_pkg/mod2.py`

However, the script syntax does work properly with absolute paths if if you set the enviornment variable `PYTHONPATH=.` (I don't know about relative paths - I don't use those). That would presumably allow pdb to work (but, shame on me, I've never tried it).

Re: CLI tools hidden in the Python standard library

#53

Post by NoboruWataya »

> I thought this might provide a utility for generating random numbers, but sadly it's just a benchmarking suite with no additional command-line options:

I had the same experience with the tkinter ones - I thought they might be like zenity, a way to build simple UI elements from the command line. But they mostly just show simple non-configurable test widgets. The colour chooser could be helpful though.

Re: CLI tools hidden in the Python standard library

#54

Post by jon-wood »

tomrod wrote: post #50
p4bl0 wrote: post #44

I often use `python -m http.server` e.g. to easily share files over local networks but I had no idea so many standard modules supported this. Thanks for sharing this link!

Do you have an example snippet you could share?

`python3 -m http.server --help` - its really not that complex.

Re: CLI tools hidden in the Python standard library

#55

Post by quenix »

tomrod wrote: post #50
p4bl0 wrote: post #44

I often use `python -m http.server` e.g. to easily share files over local networks but I had no idea so many standard modules supported this. Thanks for sharing this link!

Do you have an example snippet you could share?

You just start an http server with

    python3 -m http.server 8080
and then access the server on the receiving device through its IP address. It’ll show a basic directory listing, and you can download from there.

Re: CLI tools hidden in the Python standard library

#56

Post by LtWorf »

trotro wrote: post #8
CJefferson wrote: post #3

I use http.server all the time, particularly as modern browsers disable a bunch of functionality if you open file URLs. Had no idea there was so much other stuff here!

Same here, it's also by far the most convenient way I've found to share files between devices on my network.

I wrote this, a while ago https://ltworf.github.io/weborf/qweborf.html

On plasma it also installs a right click shortcut to share directory from dolphin.

Re: CLI tools hidden in the Python standard library

#57

Post by maleldil »

andreareina wrote: post #16
BiteCode_dev wrote: post #11

Earlier quoted context omitted.

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

I've taken to adding a --pdb option to my scripts.

What does it do? Just `import pdb; pdb.set_trace()`?

Or `breakpoint()` after 3.7 (better because the user can override pdb with ipdb or other with `PYTHONBREAKPOINT`)?

Re: CLI tools hidden in the Python standard library

#58

Post by nmstoker »

carlossouza wrote: post #47
BoppreH wrote: post #22

Speaking of hidden Python tools, I'm a big fan of re.Scanner[0]. It's a regex-based tokenizer[1] in the `re` module, that for reasons is completely missing from any official documentation. You give it a pattern for each token type, and a function to be called on each match, and you get back a list of processed tokens. Importantly, it processes the list in one pass and ensures the matches are contiguous, where a naive…

This is one of the best ChatGPT use cases: creating regex complex patterns

It's excellent at both producing them and explaining what it has generated.

And they're now just regurgitated things from the web, I've had novel ones generated fine (obviously you need to test them carefully still)

Re: CLI tools hidden in the Python standard library

#59

Post by maleldil »

polyrand wrote: post #13

Python 3.12 will include a SQLite CLI/REPL in the standard library too[0][1]. This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI. [0]: https://github.com/python/cpython/blob/3fb7c608e5764559a718c... [1]: https://docs.python.org/3.12/library/sqlite3.html#command-li...

It's weird that they're adding code to the stdlib without type annotations.
Post reply on HN