Live data from Hacker News

Linux utils that you might not know

shiroyasha.io

41–50 of 159 posts

Re: Linux utils that you might not know

#41
post #22

I was really happy about tac when I found out about it, same with paste and comm as already mentioned. Gotten a lot of mileage out of the various seldom used options of uniq, diff and cut too.

I have a friend who uses `tac | tac` in pipelines to make the pipeline wait for all the data before continuing, which is an interesting hack.

Re: Linux utils that you might not know

#42
post #27
post #13

Earlier quoted context omitted.

Actually not, since these GNU tools could be used on BSD as well.

GNU tools? Original column and factor is older than the GNU project. Don't attribute everything to that bearded madman. GNU is mostly just a copycat of already free software.

Many GNU utilities provide extensions above and beyond the incredibly limited POSIX spec or more featured BSD, Solaris, etc utilities.

Though a better title for this particular post would be "POSIX" utilities.

> Don't attribute everything to that bearded madman.

And yet the bearded madman has done a lot more than you probably have.

Re: Linux utils that you might not know

#43
post #38
post #22

I was really happy about tac when I found out about it, same with paste and comm as already mentioned. Gotten a lot of mileage out of the various seldom used options of uniq, diff and cut too.

tac, but also rev. I can't think of an example right now, but that's been useful as well.

I often use rev if I need something from the end of log lines. Rev, cut field by delimiter, rev back

Re: Linux utils that you might not know

#45
post #26

Earlier quoted context omitted.

Many people chain ps and grep... pgrep does that

That's not a great example as `pgrep` is a little more nuanced than running `ps [options] | grep [string]`, eg you cannot use many of the same `ps` flags `pgrep` like you can with `ps`. At least with "in appropriate use of cat" (as some call it) you're literally just swapping the stdin file stream with a disk io file stream so there's no functional difference what-so-ever. I'm not saying I agree with the GP either th…

ps | grep | grep -v grep

amirite?

pgrep often fails me I think mostly for apps the edit their cmdline and it needs to whole word match I think?

Re: Linux utils that you might not know

#46
post #26

Earlier quoted context omitted.

Many people chain ps and grep... pgrep does that

That's not a great example as `pgrep` is a little more nuanced than running `ps [options] | grep [string]`, eg you cannot use many of the same `ps` flags `pgrep` like you can with `ps`. At least with "in appropriate use of cat" (as some call it) you're literally just swapping the stdin file stream with a disk io file stream so there's no functional difference what-so-ever. I'm not saying I agree with the GP either th…

> it's certainly a more logical program flow for a human to parse. ie "open file, grep for contents, do something else, etc".

If you mean adding a 'cat' at the start to provide a clear entry-point for the data, you can actually put the redirect at the beginning:

cheesy_lines.txt

Re: Linux utils that you might not know

#48

* 'comm -3' is a quick way to do a set difference from the command line. * Newer versions of sort have an option for running sorts in parallel. If you're using an older sort, you can split the files, sort them individually with GNU parallel, and use --merge to combine them. * If you have scripts that read data files, process them, and output more files, consider using a Makefile. * tmux is a good way to leave a devel…

> If you have scripts that read data files, process them, and output more files, consider using a Makefile.

Dear sweet hypnotoad, don't use Makefiles for this. Scripts in a pipeline are perfectly well suited for ETL. They have the advantage of using the same language as the command language (Makefile is not shell, and when it differs it's a significant surprise). Plus, you can drop a script into a dir in your PATH and it will work in any project.

Use make if you have a project which expensively computes assets and reusing partial outputs is possible.

> pushd and popd are useful in shell scripts to temporarily change directories without forgetting where you were.

I would start with

  cd - # go back to where I just came from
Which handles most of the pushd/popd use cases (outside of writing a script).

Really, there's a whole separate set of skills for effective interactive shell use & writing great bash scripts (e.g. parameter expansion and history manipulation).

Re: Linux utils that you might not know

#49
post #40

It always pains me when I see people use "cat" left and right, even when they don't need it. This makes for good reading: http://porkmail.org/era/unix/award.html

Once I saw this notation, I immediately fell in love: output.txt

You get a blank screen, but if you do:

cat file-that-doesnt-exist.txt | less

You get a nice error message:

cat: 'file-that-doesnt-exist.txt': No such file or directory

Post reply on HN