Useful one-line scripts for sed (2005)

#1

Post by kawera » Wed, Jun 07, 2017, 5:22 PM UTC

Useful one-line scripts for sed (2005)
pement.org

Re: Useful one-line scripts for sed (2005)

#2

Post by baldfat » Thu, Jun 08, 2017, 2:38 AM UTC

I don't know what wins for most under used tool, sed or awk?

Both of these do wonders for clearing up messy data and for a wealth of other things.

Re: Useful one-line scripts for sed (2005)

#3

Post by gwaefawfewaef » Thu, Jun 08, 2017, 3:03 AM UTC

This guy is very interesting. pement.org

http://pement.org/perl/biblink126_pl.txt

Re: Useful one-line scripts for sed (2005)

#4

Post by bpchaps » Thu, Jun 08, 2017, 3:39 AM UTC

gwaefawfewaef wrote: post #3

This guy is very interesting. pement.org http://pement.org/perl/biblink126_pl.txt

Definitely. I heard about him (10 years ago) from one of his old bosses who mentioned this page. This sed one liner page took about a month to go through from basically zero Unix experience. Probably the most important learning piece I've ever gone through. Good stuff.

Re: Useful one-line scripts for sed (2005)

#5

Post by zurn » Thu, Jun 08, 2017, 5:36 AM UTC

Widely useful pick of the bunch:

  # print section of file between two regular expressions (inclusive)
  sed -n '/Iowa/,/Montana/p'             # case sensitive

Re: Useful one-line scripts for sed (2005)

#6

Post by predakanga » Thu, Jun 08, 2017, 6:51 AM UTC

zurn wrote: post #5

Widely useful pick of the bunch: # print section of file between two regular expressions (inclusive) sed -n '/Iowa/,/Montana/p' # case sensitive

I've found that pattern to be very useful when altering/sanitising MySQL backups.

  # Set the engine of all tables in schema `foo` to TokuDB
  sed -re '/^USE `foo`/,/^USE / s/^\) ENGINE=\S+/) ENGINE=TokuDB/'

Re: Useful one-line scripts for sed (2005)

#7

Post by 801699 » Thu, Jun 08, 2017, 7:25 AM UTC

Here is something to use as a replacement for "grep -q".

In the past I found grep -q was not portable.

There are times when I do not have grep.

But I always have sed.

   cat grepq

   test $# -ge 1||
   exec echo usage: $0 PATTERN \[FILE\]

   # count lines until PATTERN 
   __=$(exec sed -n '/'"$1"'/!d;=;q' $2);
   # no of lines 
   exec test ${#__} -gt 0;

For example,

   grepq '93.184.216.34 example.com' /etc/hosts||
   echo 93.184.216.34 example.com >> /etc/hosts

One of the very early OReilly books has a chapter or two about sed that tries to describe it using an analogy to a scrivener in a monastary. Quite amusing. This is not the "sed and awk" book. It was an earlier book on text editing with UNIX. It may have been co-authored by OReilly himself; I cannot remember.

Re: Useful one-line scripts for sed (2005)

#8

Post by majewsky » Thu, Jun 08, 2017, 8:47 AM UTC

zurn wrote: post #5

Widely useful pick of the bunch: # print section of file between two regular expressions (inclusive) sed -n '/Iowa/,/Montana/p' # case sensitive

I use stuff like that all the time with config files where sections span multiple lines. For example, if ~/.ssh/config has a section like

  Host example
    HostName example.com
    IdentityFile ~/.ssh/id_example
    Port 1234

  Host next
    ...

Then the following selects the identity file:

  sed -n '/^Host example/,/^Host\>/{/^\s*IdentityFile/{s/^\s*IdentityFile\s*//;p}}' ~/.ssh/config

Although I would probably use awk after sed because it handled space-separated fields nicer:

  sed -n '/^Host example/,/^Host\>/p' ~/.ssh/config | awk '$1=="IdentityFile"{print$2}'