Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New(ish) command line tools

jvns.ca

Post reply on HN
Print view View original

191–200 of 252 posts

Re: New(ish) command line tools

#191

Post by tstack »

francislavoie wrote: post #178
mihaitodor wrote: post #86

Earlier quoted context omitted.

Speaking of logs, angle-grinder is amazing: https://github.com/rcoh/angle-grinder

That looks awesome. Something I'd love help with, I use pdsh to tail logs from multiple servers at once, but the ways I can manipulate the logs feel really limited because of how pdsh works. Does anyone know of a better solution for that? Like, from a head node, aggregate/tail the contents of the same log file on multiple servers. Bonus points if it uses 'genders' too to get the list of servers.

The Logfile Navigator (https://lnav.org), a TUI for viewing log files, has support for opening logs on remote hosts via SSH in its latest version. See the following post for more information: https://lnav.org/2021/05/03/tailing-remote-files.html

As for integrating with a “genders” host DB, there’s no direct support for it. But, lnav is scriptable, so I’m pretty sure it’s possible to write a script that does what you want. I can help with that if post in the github discussion: https://github.com/tstack/lnav/discussions

Re: New(ish) command line tools

#192

Post by bloopernova »

AmpsterMan wrote: post #127
divbzero wrote: post #59

A simple trick I only figured recently is following logs with fuzzy search: tail -f /var/log/foo.log | fzf +s Or something similar for output from a dev server: make serve | fzf --ansi +s

fzf is a wonderful little tool. I use this script so much: git branch | fzf | xargs git checkout

Related git+fzf:

    [alias]
      fza = "!git ls-files -m -o --exclude-standard | fzf -m --print0 | xargs -0 git add"
Then if you run "git fza" you'll get a list of changed files in your repo, which you can use TAB to select/deselect. Hit enter and the selected files will be staged, ready for a commit. Extra cool is that it works from any subdirectory of your repo because it always lists files from the root of the repository. It's really useful for selectively adding lots of files from the command line.

I then alias "ga" to "git fza" for even less typing :)

Re: New(ish) command line tools

#193

Post by tstack »

omair_inam wrote: post #179

My personal go-to for intelligently tailing files is lnav ( https://github.com/tstack/lnav ) tho it has crashed a couple of times for me when applying a bunch of filters, etc. Is anyone aware of any other comparable shell tool for tailing a set of logs?

Sorry for the crashes, have you sent the crash logs to support@lnav.org or opened an issue on https://github.com/tstack/lnav/issues ? I try to take a look at crash reports, but I’m not always able to figure out the issue from just the logs. If you have some time to spare in replicating the problem, I can take a deeper look.

Re: New(ish) command line tools

#194

Post by philsnow »

hddqsb wrote: post #139
philsnow wrote: post #130

Earlier quoted context omitted.

Also while inside less: & shows you only lines which match a pattern. You can hit & multiple times and less will show you only those lines which match every input pattern. A ^N right after & negates the pattern. & respects the -I switch (case insensitive pattern matching). I use this all the time, especially when I'm on a machine that I don't want to bother installing something like fzf on.

Good tips. Small correction: `&` only filters by the most recent pattern.

Seems to depend on your particular less, see the last sentence of this paragraph from `man less` on my machine:

       &pattern
              Display only lines which match the pattern; lines which do not
              match the pattern are not displayed.  If pattern is empty (if
              you type & immediately followed by ENTER), any filtering is
              turned off, and all lines are displayed.  While filtering is in
              effect, an ampersand is displayed at the beginning of the
              prompt, as a reminder that some lines in the file may be
              hidden.  Multiple & commands may be entered, in which case only
              lines which match all of the patterns will be displayed.
Indeed, I thought it would behave like you describe.. when I was refreshing my memory of how the negative pattern filtering worked, I first did &/pattern and then &/^Npattern, and was surprised to see that it displayed zero matching lines.

this is my version of less:

  % less --version
  less 581.2 (POSIX regular expressions)
  Copyright (C) 1984-2021  Mark Nudelman

Re: New(ish) command line tools

#196

Post by nojito »

1vuio0pswjnm7 wrote: post #102

None of the "replacements" interest me. When I try the "new" ones I cannot get interested in them either. When trying new programs I am looking for whether a program can do something essential that I cannot do myself, e.g., with shell scripts. However I am beginning to think another reason is that these "new" utilities are consitently too complicated, e.g., too many options. It often seems as if the authors are tryin…

Classic UNIX tools are god awful to use in terms of quality of life. Newer tools are far more ergonomically pleasing.

The real reason people are hesitant to use new tools is that there is a massive switching cost for most of them.

Re: New(ish) command line tools

#197

Post by jwilk »

omair_inam wrote: post #189
michaelmior wrote: post #171

My favourite non-standard tool is pv[0]. In it's simplest use case, pv is a replacement for cat that also outputs a progress bar to stderr. You can use it to add a progress bar (or multiple) to just about any pipeline. I love it because I know if the one-liner I wrote is about to finish in a couple minutes or if it will be a while and I should either write something more efficient, or do something else while I'm wait…

From the PV man page ( https://linux.die.net/man/1/pv ) > A more complicated example using numeric output to feed into the dialog(1) > program for a full-screen progress display: ... > Frequent use of this third form is not recommended as it may cause the programmer to overheat. ^- is that a wisecrack hidden in an otherwise serious man page?

PSA: don't use linux.die.net. It's horribly outdated.

This sentence was removed from the man page in pv 1.4.0, released in December 2012.

Re: New(ish) command line tools

#199

Post by hddqsb »

philsnow wrote: post #194
hddqsb wrote: post #139

Earlier quoted context omitted.

Good tips. Small correction: `&` only filters by the most recent pattern.

Seems to depend on your particular less, see the last sentence of this paragraph from `man less` on my machine: &pattern Display only lines which match the pattern; lines which do not match the pattern are not displayed. If pattern is empty (if you type & immediately followed by ENTER), any filtering is turned off, and all lines are displayed. While filtering is in effect, an ampersand is displayed at the beginning o…

Oh wow, that's a pretty new feature! It was added in v569, 2020[1].

[1]: https://github.com/gwsw/less/commit/6a070fc53799fb86e0fe3880...

Re: New(ish) command line tools

#200

Post by jwilk »

Erwin wrote: post #156
dystroy wrote: post #138

I really like the approach of pipe-rename ( https://github.com/marcusbuffett/pipe-rename ) for renaming many files. It's especially convenient for people who can efficiently edit many lines in their favorite text editor (so everybody here, I guess). Main problem: you maybe don't do this kind of operation frequently enough to remember how it's called or how you aliased it.

I like that idea of converting metadata to text, so you can do text operations on it that you are already familiar with, and then apply the changes back. If your problem can be solved by just regexps, there's a "rename" tool installed by Perl I've used for decades. E.g. rename -n 's/JPEG$/jpg/' *JPEG Will apply the substitution for JPEG at end of the filename to just jpg. The -n will run without renaming so you can s…

Beware there are two incompatible rename programs in the wild. :(

What you have installed, and what Debian and its derivates ship is this: https://metacpan.org/pod/distribution/File-Rename/rename.PL

Most(?) other distros ship the one from util-linux: http://man7.org/linux/man-pages/man1/rename.1.html

Post reply on HN