Live data from Hacker News

New(ish) command line tools

jvns.ca

221–230 of 252 posts

Re: New(ish) command line tools

#221

Earlier quoted context omitted.

ls | % {rename-item $_ -newname $('myfile_' + ($_.BaseName -replace '\D') + '.md' )} No need to remember anything.

? I would be much more likely to remeber ls | renamer than figure out how to write such a command.

> ?

Nope.

> to remeber

That's the point, you don't need to remember anything, it would come naturally after a day of tinkering with PS/pwsh.

You just write it for your current needs.

This example was just a quick one-liner, it's very simple:

    ls | % {            # get the file list, cycle for each item
    
    rename-item         # obvious usage: rename-item oldname newname
    
        $_              # pretty obvious, the source - place the loop variable here ie file name
      
        -newname        # explicit calling for a parameter name, isn't really needed 
        
          $(            # subroutine start or eval if you prefer, to construct the new file name:
                        # 'text' + (take property basename from the file and regex it) + 'text'
                        
            'myfile_' + ($_.BaseName -replace '\D') + '.md'
            
          )             # subroutine end
          
      }                 # cycle end.
It could be simplified (in this example) to this:

    ls | % {
        $a = 'myfile_' + ($_ -replace '\D') + '.md'
        rename-item $_ $a
        }
which could be even written as a one-liner if you prefer it right now, just add a semicolon after `$a = ...`

    ls | % {$a = 'myfile_' + ($_ -replace '\D') + '.md'; rename-item $_ $a }

Re: New(ish) command line tools

#222

Nice list of new tools to take a look at. However I think there's just one big drawback: as you move around different systems is very unlikely that you have those (you might even not have permission to touch those systems) so you will have to rely on the standard existing tools (even among different OSes you have slightly different implementations). That's why I lately I've investing more time reading man pages of so…

This. With companies getting ever security conscious installing any software is a long and tiresome exercise.

Re: New(ish) command line tools

#223

It's not new but I like "Miller": > Miller is a command-line tool for querying, shaping, and reformatting data files in various formats including CSV, TSV, JSON, and JSON Lines. https://miller.readthedocs.io/en/latest/

I got very excited about this tool. It's exactly what I need. I wrote a custom calendar app in it. But I got disappointed by performance. A list of 10 items takes 3 seconds to draw. I yet have to check if that is not caused by some other tool in the call stack. I wonder what is others people experience with performance of mlr?

If you can reproduce that reliably you should open a bug for it, eh? I'm sure John Kerl would want to know.

Re: New(ish) command line tools

#224

I made a tool back in college called “line” for outputting ranges of line or column numbers. I got tired of piping head into tail and found it simpler. Examples: line file.txt 5 to 9 cat file.txt | line —column 4 to 20 I thought “line” was very Unix sounding and kinda cute, but like a lot of these projects would never make its way into the gnu utils so I thought what’s the point. That and of course to a beginner Awk…

It's fun to write your own tools! (Scratch your own itch and all that.)

It sounds like your "line" replicates a use of sed that I use all the time, printing a contiguous range of lines.

The example:

line file.txt 5 to 9

can be:

sed -n '5,9p' file.txt

You mention using awk, which totally works, but to me is much less ergonomic.

You don't explain what line's "--column" option does so I'm not sure what the equivalent of that might be. That might be where awk comes into its own ... :)

> [...] and printing lines between matching words

This is the same sed command as above, but using regular expressions for the address part:

sed -n '/^func doit/,/^}$/'

will print just the function called "doit" (in properly formatted go).

Could you (I mean "one") design a "friendlier" (or more "beginner friendly") user interface than sed presents? Yes, obviously (you did exactly that). But unlocking the power (or even just beginning to "unlock" the power) of the standard tools (sed, awk, grep, tr, cut, paste, find, xargs, ...) can get you a really long way. Of course, the initial problem is how to know that one of those tools can solve the problem you have in your head.

("Bonus" sed content: replace "head":

Instead of

head -n 5

do

sed 5q

To replace tail you need tac (or "tail -r", haha))

Re: New(ish) command line tools

#226

I really like the approach of pipe-rename ( https://github.com/marcusbuffett/pipe-rename ) for renaming many files. It's especially convenient for people who can efficiently edit many lines in their favorite text editor (so everybody here, I guess). Main problem: you maybe don't do this kind of operation frequently enough to remember how it's called or how you aliased it.

" ... renaming many files ..."

I think folks should be aware of 'vimv' which opens your pwd in an editor and you just edit your filenames in 'vi' ... and all of the renaming happens upon saving and quitting.

Not sure if this is the official/canonical distribution, but for what it's worth:

https://github.com/thameera/vimv

Re: New(ish) command line tools

#227

I made a tool back in college called “line” for outputting ranges of line or column numbers. I got tired of piping head into tail and found it simpler. Examples: line file.txt 5 to 9 cat file.txt | line —column 4 to 20 I thought “line” was very Unix sounding and kinda cute, but like a lot of these projects would never make its way into the gnu utils so I thought what’s the point. That and of course to a beginner Awk…

It's fun to write your own tools! (Scratch your own itch and all that.) It sounds like your "line" replicates a use of sed that I use all the time, printing a contiguous range of lines. The example: line file.txt 5 to 9 can be: sed -n '5,9p' file.txt You mention using awk, which totally works, but to me is much less ergonomic. You don't explain what line's "--column" option does so I'm not sure what the equivalent of…

> You don't explain what line's "--column" option does

It was simply the equivalent of Awk printing columns like this.

awk '{ print $2, $3, $4 }'

Except with support for column ranges, which I don't think Awk supports without writing a for loop.

Something convoluted like :

awk -v start=1 -v end=3 '{ for (i=start; iIdk, the amount of Unix that GNU and BSD accepted as a bare environment, the range is single purpose programs like tail and mini interpreters which are powerful but require a lot of skill. If I worked in system administration, and I HAD to use shell, I'd hold onto Awk for dear life. But I don't, so it's sort of this ancient swiss army knife.

Re: New(ish) command line tools

#228

Earlier quoted context omitted.

You don’t have to stack -v like that: -v inverts all the -e expressions, so this should be equivalent: grep -inHv -e foo -e bar -e baz

So, awk sort of works as a fancy grep: ps axjw | awk '(/zsh/ || /login/) && !(/awk/ || /direnv/)' And you can easily add features like "print the first line unconditionally: ps axjw | awk 'NR == 1 {print} (/zsh/ || /login/) && !(/awk/ || /direnv/)' IMO, awk is in a pretty uniquely sweet spot between grep and perl/ruby/python.

Thanks for the grep tip!

Unfortunately I've not put in the effort to learn awk past printing the n-th column of ls (my typical use) - the extra syntax required to properly 'quote' and {} things puts me off.

Re: New(ish) command line tools

#229
post #193

My personal go-to for intelligently tailing files is lnav ( https://github.com/tstack/lnav ) tho it has crashed a couple of times for me when applying a bunch of filters, etc. Is anyone aware of any other comparable shell tool for tailing a set of logs?

Sorry for the crashes, have you sent the crash logs to support@lnav.org or opened an issue on https://github.com/tstack/lnav/issues ? I try to take a look at crash reports, but I’m not always able to figure out the issue from just the logs. If you have some time to spare in replicating the problem, I can take a deeper look.

I do try to submit the logs when I can. tbh, it takes a combination of (In/Out) text filters along with some navigation to trigger the failures. If I re-encounter, I will try to spend some time to reproduce.

You can find the last failure I submitted here[0].

[0] = https://groups.google.com/g/lnav/c/C2uORuyKmy8/m/5Knove9gBQA...

Post reply on HN