Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Useful sed scripts and patterns

github.com

Post reply on HN
Print view View original

81–90 of 124 posts

Re: Useful sed scripts and patterns

#81

Post by zouhair »

baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

In these instances where you find a one liner useful but you use it rarely, just put it in a function with comments on how it works. When you want to use it the function is ready and the explanation too.

Re: Useful sed scripts and patterns

#82

Post by krinchan »

I use sed -ibak ‘10d’ ~/.ssh/.known_hosts so many times a week. awk, sed, and regex really are the two programs I am happy to have learned well at my first job. They’ve helped me immensely throughout my 10 years as someone who bridges developer and ops roles.

Re: Useful sed scripts and patterns

#83

Post by chasil »

adrianscheff wrote: post #43
saurik wrote: post #9

There are a ton of little errors in this post (at the time of my leaving this comment, of course): the variable delimiter example is missing the close _, the first word example is missing the open ', and the except line 5 example is missing a space; meanwhile, these snippets will cause bad habits, such as thinking about "words" as "things involving letters", using -r instead of -E, and a massive knowledge gap with re…

Thank you for the feedback and the constructive criticism. Regarding the hold space I found it very confusing to use and understand. IMO it represents the 80/20 of sed (80% effort for 20% results). You're right, I'm using -r even when it's not necessary. To my defense I think it's a good habit to have since without it regex expressions are painful to write. I didn't considered that using -E it's a better choice but I…

CentOS 7's sed doesn't support -E, but CentOS 8 does. That might cause some confusion.

Also, FYI, this is the link for the POSIX specification for sed:

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s...

Notice that -i is not POSIX.

Re: Useful sed scripts and patterns

#85

Post by jaimehrubiks »

What is the easiest way to make sure a "field, or option" is present in a file. Similar to what ansible's lineinfile does?

For example suppose I need to add this setting to /etc/ssh/sshd_config:

PermitRootLogin yes

One idea that comes to mind is to first delete the line with the line shown in this post, and then add it back:

Regex sed -E '/^#/d' file.txt - delete lines where regex matches

sed -E '/^PermiteRootLogin/d' /etc/ssh/sshd_config ; echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config

Is there a more straightforward way with sed?

Re: Useful sed scripts and patterns

#86

Post by SAI_Peregrinus »

drran wrote: post #62
undershirt wrote: post #24

Is there a way we ask these kinds of question in plain english? and then have it return a snippet like this

StackOverflow

StackOverflow: "There are no stupid questions, but there are a lot of inquisitive idiots. Closing as duplicate of ."

Re: Useful sed scripts and patterns

#87

Post by chasil »

baby wrote: post #32
Debug_Overload wrote: post #26

Earlier quoted context omitted.

How long does the relearning take exactly? My experience has been that if you've used these tools before, it's pretty easy to remember or refresh your memory. In that sense, they're not very different from other tools/syntaxes that you haven't used in a while. And if you do use them frequently enough, you might not even need to refresh/relearn anything.

I basically gave up on relearning them considering the number of times I’ve had to do it and the amount of effort involved to relearn them. Maybe it’s just me, but I just write python now.

The full specification for the AWK language is 40 pages in the book written by the authors.

I sometimes forget the exact syntax for pipes and file open/close, but it's not hard to find in 40 pages.

AWK is in busybox, and a lot of other places that Python simply cannot go.

The control structure syntax is also the same a C/JavaScript/PHP/C++ etc., so it certainly does not hurt to know it.

https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/The_AWK_P...

Re: Useful sed scripts and patterns

#88

Post by chasil »

sam_lowry_ wrote: post #36
baby wrote: post #32

Earlier quoted context omitted.

I basically gave up on relearning them considering the number of times I’ve had to do it and the amount of effort involved to relearn them. Maybe it’s just me, but I just write python now.

You need to have a mental model for each tool and some idea of syntax associated with it. Once I learned about IFS,OFS, BEGIN, END and {}, as well as the fact that variables and their references have the same syntax, unlike in Perl... I could pick up coding in awk at any moment

IFS is the POSIX shell; in awk, it's just FS.

I think by {} you mean an argument to the find command.

Re: Useful sed scripts and patterns

#89

Post by twic »

WastingMyTime89 wrote: post #70
twic wrote: post #57

Earlier quoted context omitted.

> I think just learning how sed works is actually easier than treating it like magic. It's not awk. As someone with over a decade of heavy sed use, who only got into awk recently, i'd say awk is far easier and less magical. Awk is just another programming language, for the most part. Using sed for anything substantial requires you to think in strange directions, to find ways to thread state and control flow through t…

I think I both agree and disagree. As you rightfully pointed, I wouldn’t do anything overly complicated with sed because it really is a line editor and anything which is not line based and can’t be explained simply in the context of using an editor will quickly seem clunky and overly complicated. I wouldn’t do anything period with awk. I agree with you that it’s just another programming language but one which feels e…

Interesting! I don't find awk too strange; indeed i strongly prefer it to Perl, which i wrote a lot of around the turn of the century.

These days i reach for awk for a lot of log processing, particularly when it needs to be stateful. For example, i have a log file with entries for remote clients connecting and disconnecting. An awk script can loop over that and keep an array of clients, adding and removing them as appropriate, and be easily tweaked to report different things - emit an event for each connection-disconnection pair, or track the number of clients connected over time, etc. I couldn't do that at all with sed. It would be more awkward with bash. It would be easy enough, if a little bit more verbose, in Python.

Re: Useful sed scripts and patterns

#90

Post by pnutjam »

sergiosgc wrote: post #54
sitkack wrote: post #29

Earlier quoted context omitted.

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…

> Awk/sed made sense for the time and place they are from, but things have changed. Don't do that. Do not assume something is of no use because you don't know how to use it. Double that recommendation for tools that have widespread use. There is a reason they're used everywhere. In the case of awk/sed/perl: Nothing beats them for composition in one-liner text processing pipelines.

plus, they exist in the vast majority of unix like systems. You don't have to install anything. They are just there and available to use. Even using them in a simplistic manner is light-years ahead of trying to do stuff manually.
Post reply on HN