What is the easiest way to make sure a "field, or option" is present in a file. Similar to what ansible's lineinfile does? For example suppose I need to add this setting to /etc/ssh/sshd_config: PermitRootLogin yes One idea that comes to mind is to first delete the line with the line shown in this post, and then add it back: Regex sed -E '/^#/d' file.txt - delete lines where regex matches sed -E '/^PermiteRootLogin/d…
Useful sed scripts and patterns
101–110 of 124 posts
Re: Useful sed scripts and patterns
#102I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
100%. The other day I wanted to do some transformations on a file using the like of regex, sed and grep. After 1 hour I just gave up and thought "hey, why not try doing it in Python". It took me just 5 minutes to write a small script, which was eye opening. Since then, I just write python scripts instead of toiling with commands I don't know.
Re: Useful sed scripts and patterns
#103Mastering one tool is probably wise, but it's also good to know which tools may be better suited for some purposes. For "keep the first word of every line", I prefer awk: awk '{ print $1 }'
I agree with the people here saying that it's better to just use your favourite scripting language unless your job is mostly about writing little bash scripts like that (maybe sysops people?)... Here's a groovy script to do the same: new File('file.txt').eachLine { println it.split()[0] } Not as neat as awk, but not much worse either... and as I use Groovy a lot for testing, and its syntax is simplified Java (which I…
I adore Groovy the language but am compelled to note its horribly slow bootup time. On a commodity laptop it's usually a couple of seconds - acceptable for some things, but hard to justify for one-liners.
Re: Useful sed scripts and patterns
#104Earlier quoted context omitted.
If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…
Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?
Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new.
(I might be wrong about "everything" but that's what bomb-tossing is all about)
Re: Useful sed scripts and patterns
#105I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget.
I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades are the same way. I have a terrible memory, but it turns out that does not have to be a big hindrance for tech work.
When I'm working (or tinkering at home), I keep a tab to my personal wiki open at all times. I will not usually bother to write something down the first time I do it. But I have to google it twice, I throw it into my wiki. This does two things: 1) It makes me (slightly) more likely to actually remember it for next time, and 2) The next time I need the information, I know where I can find it without having to wade through SEO-encumbered blog spam or outdated StackOverbutt answers.
A database of personal notes is a powerful multiplier for technical ability and productivity. To the point that whenever I interview a candidate for a job, one of the things I ask is what system they keep their notes in. Their system doesn't matter at all to me but if the answer is none, that's a definite strike against.
>I really don’t think non-interactive CLIs are a good way to do complicated tasks.
Many of us thrive on the Unix command line because of the raw power you can wield with it. "Why waste time trying to learn arcane shell commands when you can just write a script in Python?" is a common refrain I hear from developers with only a few years under their belt. My response to that is, "Why would I waste time writing a Python script when I can do it in one line of shell?"
Re: Useful sed scripts and patterns
#106I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades…
I 100% agree: If I have to look something up twice, it's going in my wiki.
The first time you look something up, it might not be obvious if you'll ever use that info again, so why waste the time and clog up your notes with a bunch of noise? But if you've had to look something up twice, odds are good you'll have to look it up again someday.
And like you said, writing a note in a way that's clear and understandable helps you to recall the thing a little better next time
Re: Useful sed scripts and patterns
#107I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades…
> personal wiki open at all times
Any suggestion for a remotely accessible personal wiki? (I'm assuming that access is restricted for potential "personal" stuffs).
Re: Useful sed scripts and patterns
#108I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
This is everyone :)
I love awk and whenever I __do__ find a task where awk helps me, it's really an amazing experience and it is always a boon, never a burden.
But I really suck at remembering the syntax as I only hit tasks that awk solves well rarely. What I do remember is situations where awk definitely can likely help. Same with sed, tr, etc.
Keep in mind, every time we automate something or introduce a new tool/process, it's a cost benefit analysis; when I have dozens of gigs of logs to parse to figure out which hosts out of hundreds are having issues, it's a no-brainer for me; the 30 minutes to revisit a few awk and bash tricks is a far better investment than trying to grep/scroll my way through literally millions of lines of logs/code.
When I first learned awk I definitely went through a 'hammer' phase (i.e., when you have a hammer, everything looks like a nail); once phase passed, I find that I'm much more disciplined on when I break out awk or similar tools.
Ultimately it's about identifying when you have the "right" tool for a specific job; I might need to invest time to revisit some syntax, but if the end result is that my 30 minutes saves hours, then it's a good 30 minutes. I'm at the stage where the basic text munging with awk is a no-brainer (or at least I remember the mistakes I made previously and how I fixed them); this was achievable only after a few projects where I did have to spend a bit more time wrangling awk than I preferred, but it saved me a lot of time in the future. Just knowing what I __can__ do with a given tool helps me make educated decisions on where I should invest my time on a given issue.
Re: Useful sed scripts and patterns
#109Earlier quoted context omitted.
Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?
Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new. (I might be wrong about "everything" but that's what bomb-tossing is all about)
Welcome to the discussion. I like it, I haven't perled in a while but does that mean you can evolve from a one liner to a full program using all of perl using this technique?
Re: Useful sed scripts and patterns
#110I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.
> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades…
I do the same thing with obsidian and use the git plugin to keep it in sync (via gitlab) across everywhere that I use it. I know people who similar with .org emacs files. It works pretty well. I don't have the patience to maintain a wiki :)