CLI tools hidden in the Python standard library

Post by chmaynard » Thu, Jun 29, 2023, 6:12 PM UTC

chmaynard wrote: post #17

polyrand wrote: post #13

Python 3.12 will include a SQLite CLI/REPL in the standard library too[0][1]. This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI. [0]: https://github.com/python/cpython/blob/3fb7c608e5764559a718c... [1]: https://docs.python.org/3.12/library/sqlite3.html#command-li...

> This is useful because most operating systems have sqlite3 and python3, but are missing the SQLite CLI. Not sure what you mean -- sqlite3 is the SQLite CLI.

    The SQLite project provides a simple command-line program named sqlite3 (or sqlite3.exe on Windows)
    that allows the user to manually enter and execute SQL statements against an SQLite database
    or against a ZIP archive.

Post by patrec » Thu, Jun 29, 2023, 6:28 PM UTC

formerly_proven wrote: post #14

mg wrote: post #10

There is one problem with those: Security. Using modules (even if they are in the standard library) on the command line lets malicious code in the current dir take over your machine: https://twitter.com/marekgibney/status/1598706464583028736

iirc this is one of the things earmarked for a hypothetical Python 4, making -P the default. It's also one of the many relatively well-known (security) issues in Python that don't get addressed for a surprising amount of time. Others in the same vein would be stuff like stderr being block-buffered when not using a TTY, no randomized hashes for the longest time, loading DLLs preferably from the working directory, std*…

Post by johnnymellor » Thu, Jun 29, 2023, 6:47 PM UTC

johnnymellor wrote: post #94

sharikous wrote: post #51

Earlier quoted context omitted.

Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm? Multiline statements are not accepted, nor things like if/for Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab ar…

Not supporting multi-line statements is just because pdb doesn't bother to parse the statement to work out if it is an incomplete multi-line statement. That could be easily fixed (I have a prototype patch for that using `code.compile_command`). The scope problems are more fundamental: The pdb REPL calls[1] the exec builtin as `exec(code, frame.f_globals, frame.f_locals)`, which https://docs.python.org/3/library/funct…

Post by bdzr » Thu, Jun 29, 2023, 8:02 PM UTC

xorcist wrote: post #119

knodi123 wrote: post #103

Earlier quoted context omitted.

not this? grep -ve 'test/|tests/|idlelib|turtledemo'

That would be -vE, or you have to escape the pipe symbols.

Post by fastasucan » Thu, Jun 29, 2023, 8:34 PM UTC

burntsushi wrote: post #82

BoppreH wrote: post #22

Speaking of hidden Python tools, I'm a big fan of re.Scanner[0]. It's a regex-based tokenizer[1] in the `re` module, that for reasons is completely missing from any official documentation. You give it a pattern for each token type, and a function to be called on each match, and you get back a list of processed tokens. Importantly, it processes the list in one pass and ensures the matches are contiguous, where a naive…

You'll be able to do this soon with the Rust regex crate as well. Well, by using one of its dependencies. Once regex 1.9 is out, you'll be able to do this with regex-automata: use regex_automata::{ meta::Regex, util::iter::Searcher, Anchored, Input, }; #[derive(Clone, Copy, Debug)] enum Token { Integer, Identifier, Punctuation, } fn main() { let re = Regex::new_many(&[ r"[0-9]+", r"[a-z_]+", r"[,.]+", r"\s+", ]).unwr…