Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Useful sed scripts and patterns

github.com

Post reply on HN
Print view View original

111–120 of 124 posts

Re: Useful sed scripts and patterns

#111

Post by stjohnswarts »

nomel wrote: post #107
bityard wrote: post #105

Earlier quoted context omitted.

> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades…

> personal wiki open at all times Any suggestion for a remotely accessible personal wiki? (I'm assuming that access is restricted for potential "personal" stuffs).

I'm not big on wiki's but almost everyone I know hosts one on something like vultr/digital ocean, backs up to home, and does what OP mentioned about updated it in the backgroung. Personally I use markdown files+gitlab+obsidian. there are a million ways. You might like like twiki for a "standard" wiki or tiddlyWiki for a more eccentric one :)

Re: Useful sed scripts and patterns

#112

Post by stjohnswarts »

thaumaturgy wrote: post #27
baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

I'm mostly with you (except for your last sentence). I've picked up a lot of languages over the years, and can fall back into most of them pretty fast, but sed and awk especially are hard. On the occasions that they really are the best tool for the job, I get actual books out and set aside a window of a few hours to plod through it. I think the problem is that they fall into an uncanny valley between "simple tool" an…

I gave up on them except for the very most common uses of them for slicing/dicing text, after the most essential basics I just turn to python and suck it up. Learning to pipe things in/out in python is a great skill to have in these circumstances. I think perl is the king of this but I'm not going back to that... never again :)

Re: Useful sed scripts and patterns

#113

Post by stjohnswarts »

baby wrote: post #31
sitkack wrote: post #29

Earlier quoted context omitted.

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…

I really wish all these bash scripts would be replaced with python scripts.

Yeah I redid some scripts on my team for my own usage because our team bash guru maintained them and didn't want anyone to change them. So I did my own versions in python. Sure they were mostly 2x+ as long but they were very easy to read and modify and test. The bash guru finds out and brings it up at a team meeting trying to shame me, but it ended up with my boss asking the team, and they voted on using my scripts in lieu of the older bash scripts as I had written a few unit tests around them and the rest of the team understood them without digging out unix tomes.

Re: Useful sed scripts and patterns

#114

Post by sitkack »

stjohnswarts wrote: post #113
baby wrote: post #31

Earlier quoted context omitted.

I really wish all these bash scripts would be replaced with python scripts.

Yeah I redid some scripts on my team for my own usage because our team bash guru maintained them and didn't want anyone to change them. So I did my own versions in python. Sure they were mostly 2x+ as long but they were very easy to read and modify and test. The bash guru finds out and brings it up at a team meeting trying to shame me, but it ended up with my boss asking the team, and they voted on using my scripts i…

I have done something similar, what I really found worked well was to wrap the bash scripts in python to instrument them and if they just operate on data, you can even spawn another command and run them both at the time confirming that the output matches.

Great way to have confidence in a port and provide for a fallback path. Eventually the new code will take over the old code and the old code will wither and die.

Re: Useful sed scripts and patterns

#115

Post by kerblang »

sitkack wrote: post #109
kerblang wrote: post #104

Earlier quoted context omitted.

Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new. (I might be wrong about "everything" but that's what bomb-tossing is all about)

Welcome to the discussion. I like it, I haven't perled in a while but does that mean you can evolve from a one liner to a full program using all of perl using this technique?

I suppose it's a "pathway to perl" in the sense of a mild gateway drug. Perl is definitely one-liner-friendly. Every time I get into Perl I have to relearn it all over again (which is apparently not an isolated phenomenon), so I don't have great suggestions for digging further. I just habitually reach for perl -pe about 3/4 of the time.

Re: Useful sed scripts and patterns

#116

Post by News-Dog »

s1291 wrote: post #44
baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

cheat.sh is your friend. E.g for sed: curl cheat.sh/sed Will show you several examples.

cheat.sh & cht.sh https://github.com/chubin/cheat.sh

Also, LDP: Advanced Bash-Scripting Guide: https://tldp.org/guides.html

Re: Useful sed scripts and patterns

#117

Post by jseban »

bityard wrote: post #105
baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

> I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I'm going to let you in on a little secret: many of us who have been doing development or systems administration in some form another for multiple decades…

This is what I do as well, I don't understand why programmers try so hard to avoid ever taking notes (or reading a manual). It's somehow too boring for programmers to work in a structured way, they want to pretend that their job is like playing a video game. I don't understand what the problem is of looking something up, and why is it so "bad" to read a little syntax before you get started, especially if it gives you better solutions.

Imagine having this attitude towards tools in general, you'd throw out most of everything. Anything that's powerful has a learning curve

Re: Useful sed scripts and patterns

#118

Post by sam_lowry_ »

chasil wrote: post #88
sam_lowry_ wrote: post #36

Earlier quoted context omitted.

You need to have a mental model for each tool and some idea of syntax associated with it. Once I learned about IFS,OFS, BEGIN, END and {}, as well as the fact that variables and their references have the same syntax, unlike in Perl... I could pick up coding in awk at any moment

IFS is the POSIX shell; in awk, it's just FS. I think by {} you mean an argument to the find command.

Indeed. Thanks for the hint.

Re: Useful sed scripts and patterns

#119

Post by dang »

thaumaturgy wrote: post #30
sitkack wrote: post #29

Earlier quoted context omitted.

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…

Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?

"Be kind. Don't be snarky."

"Please don't sneer, including at the rest of the community."

https://news.ycombinator.com/newsguidelines.html

Re: Useful sed scripts and patterns

#120

Post by dang »

sitkack wrote: post #35
yesenadam wrote: post #33

Earlier quoted context omitted.

I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason. Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk . I got into sed a while ago but never use it. A…

Heh, Not over here doing arm chair yucking someone's yum, to you and everyone else in this shrub of the conversation. If you guys want to do SCA or model trains with ancient programming languages, go all in. I play around with Forth and APL as well. If the premise is the title of the post "day to day usage ..." then total task efficiency is important. Are you moving the goal posts and putting words in my mouth? If I…

Would you please not post in the flamewar style to HN, regardless of how provocative another comment was or you feel it was? This is not at all what we're trying for here.

https://news.ycombinator.com/newsguidelines.html

Post reply on HN