Live data from Hacker News

CLI tools hidden in the Python standard library

til.simonwillison.net

141–150 of 160 posts

Re: CLI tools hidden in the Python standard library

#141
post #137

Earlier quoted context omitted.

> The bafflement increases. So does the pendantry.

Please don't derail discussions with zero-effort, low-substance nonsequitur oneliners like this.

Sorry you're right. But you are being pedantic. These are quick hacks that might come in handy a couple times per year, maybe. Bringing up security, alternative native tools, and even trying to find a formal definition for "most people" is, imo, missing the point.

Re: CLI tools hidden in the Python standard library

#142
post #137

Earlier quoted context omitted.

Please don't derail discussions with zero-effort, low-substance nonsequitur oneliners like this.

Sorry you're right. But you are being pedantic. These are quick hacks that might come in handy a couple times per year, maybe. Bringing up security, alternative native tools, and even trying to find a formal definition for "most people" is, imo, missing the point.

[deleted]

Re: CLI tools hidden in the Python standard library

#143

Earlier quoted context omitted.

You'll be able to do this soon with the Rust regex crate as well. Well, by using one of its dependencies. Once regex 1.9 is out, you'll be able to do this with regex-automata: use regex_automata::{ meta::Regex, util::iter::Searcher, Anchored, Input, }; #[derive(Clone, Copy, Debug)] enum Token { Integer, Identifier, Punctuation, } fn main() { let re = Regex::new_many(&[ r"[0-9]+", r"[a-z_]+", r"[,.]+", r"\s+", ]).unwr…

'A bit more verbose' = 2,5 times as many characters. Not saying its good or bad just saying its a bit more verbose ;)

Yes, as I said, the APIs exposed in regex-automata give a lot more power. It's an "expert" level crate. You could pretty easily build a scanner-like abstraction and get pretty close to the Python code.

I posted this because a lot of regex engines don't support this type of use case. Or don't support it well without having to give something up.

Re: CLI tools hidden in the Python standard library

#144
post #72

Earlier quoted context omitted.

Amazing! I had just written a regex tokenizer in python the other day, this would have been great!

Curious cat - had you considered using Antlr4 and a Python visitor/listener? (Were you aware of antlr?). Depending what you're trying to do with a regex tokenizer, it might be suitable.

It was more just for fun than anything else. I was more interested in the recursive descent parser so the tokenizer was just a step along the way.

I've played with Antlr but never really vibed with it to be honest (and it was years ago)

Re: CLI tools hidden in the Python standard library

#145
post #22

Speaking of hidden Python tools, I'm a big fan of re.Scanner[0]. It's a regex-based tokenizer[1] in the `re` module, that for reasons is completely missing from any official documentation. You give it a pattern for each token type, and a function to be called on each match, and you get back a list of processed tokens. Importantly, it processes the list in one pass and ensures the matches are contiguous, where a naive…

This is one of the best ChatGPT use cases: creating regex complex patterns

this one use case justified copilot as an expense for me.

Re: CLI tools hidden in the Python standard library

#146

Earlier quoted context omitted.

-P?

https://docs.python.org/3/using/cmdline.html?highlight=q#cmd... had to look myself. apparantly, it's like setting PYTHONSAFEPATH which prevents 'unsafe' paths from getting added to sys.path. new in 3.11

New in 3.11; ah; that's why I didn't see the option in my default python.

Re: CLI tools hidden in the Python standard library

#147
post #38

Earlier quoted context omitted.

There's no magic, only layers. `python -m pdb ` runs pdb with the rest of the arguments. pdb handles the second `-m`. If you have a fancy IDE feature, open a new python file, type "import pdb", use go to definition on pdb to jump to that file in the standard library, and read its main function - it handles -m explicitly :)

Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm? Multiline statements are not accepted, nor things like if/for Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab ar…

After triggering pdb by having "breakpoint()" in tour python code and dropping in the debugger you can type "interactive" in the console to enter multiline strings.

Re: CLI tools hidden in the Python standard library

#148

Earlier quoted context omitted.

Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm? Multiline statements are not accepted, nor things like if/for Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab ar…

After triggering pdb by having "breakpoint()" in tour python code and dropping in the debugger you can type "interactive" in the console to enter multiline strings.

TIL again. Very good thread!

It's "interact" though, not "ive".

You can exit it to carry on with the regular pdb.

Re: CLI tools hidden in the Python standard library

#149
post #99

Earlier quoted context omitted.

To be fair, the advantage of doing separate greps is working iteratively and drilling down on what you want.

You can iteratively add the -e flags just the same.

Sure, for the basic example of 5 simple greps. Usually I have some additional flags or steps in between too.

Re: CLI tools hidden in the Python standard library

#150
post #66

Earlier quoted context omitted.

This is quite an oversimplification. Some modules have no active maintainers and take time from the small team that work on CPython. Some modules are deprecated and removed to make it possible to allocate time on other improvements, and they went back on some removals when users came forward showing that they were still needed. There is some discussion about the rationale at https://peps.python.org/pep-0594/#rational…

Did you read the latest news or not? They regret asyncio! (and many other core modules). They won't remove them… but they regret having made them. I think without, people would have just not used python.

After using Trio, I would regret asyncio too
Post reply on HN