Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Useful sed scripts and patterns

github.com

Post reply on HN
Print view View original

31–40 of 124 posts

Re: Useful sed scripts and patterns

#31

Post by baby »

sitkack wrote: post #29
thaumaturgy wrote: post #27

Earlier quoted context omitted.

I'm mostly with you (except for your last sentence). I've picked up a lot of languages over the years, and can fall back into most of them pretty fast, but sed and awk especially are hard. On the occasions that they really are the best tool for the job, I get actual books out and set aside a window of a few hours to plod through it. I think the problem is that they fall into an uncanny valley between "simple tool" an…

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…

I really wish all these bash scripts would be replaced with python scripts.

Re: Useful sed scripts and patterns

#32

Post by baby »

Debug_Overload wrote: post #26
baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

How long does the relearning take exactly? My experience has been that if you've used these tools before, it's pretty easy to remember or refresh your memory. In that sense, they're not very different from other tools/syntaxes that you haven't used in a while. And if you do use them frequently enough, you might not even need to refresh/relearn anything.

I basically gave up on relearning them considering the number of times I’ve had to do it and the amount of effort involved to relearn them. Maybe it’s just me, but I just write python now.

Re: Useful sed scripts and patterns

#33

Post by yesenadam »

thaumaturgy wrote: post #30
sitkack wrote: post #29

Earlier quoted context omitted.

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the…

Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?

I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason.

Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk. I got into sed a while ago but never use it. Awk, together with sort and uniq sometimes, is all I need.

And it is interactive. You have an input. Print that out and look what needs to be done. Write the first transforming step in Awk. See what that did. Add the next step. See what that did. Extremely interactive. After a few minutes I have a one-liner (often 2 or 3 wrapped lines) that does exactly what I want, and easy to adapt to other similar things. I have very large bash history enabled so they stay Ctrl-R-accessible.

Re: Useful sed scripts and patterns

#34

Post by aidos »

baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

That’s just because you don’t use them often though. If they were a daily driver it’d be different.

I think that if you have a good mental model of how awk works, then you know times that it’s the right tool for the job. And on those occasions you’ve got plenty of time to go search for some syntax, make a coffee, maybe even go for a run all with change to spare compared to using pretty much anything else.

I will admit, that if I can do it with a few greps, seds and cuts then I tend to just use those first, because there no thinking involved for me.

Re: Useful sed scripts and patterns

#35

Post by sitkack »

yesenadam wrote: post #33
thaumaturgy wrote: post #30

Earlier quoted context omitted.

Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?

I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason. Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk . I got into sed a while ago but never use it. A…

Heh, Not over here doing arm chair yucking someone's yum, to you and everyone else in this shrub of the conversation.

If you guys want to do SCA or model trains with ancient programming languages, go all in. I play around with Forth and APL as well.

If the premise is the title of the post "day to day usage ..." then total task efficiency is important.

Are you moving the goal posts and putting words in my mouth? If I have to enumerate EPR then we also suffer from the HN Standard Reply where caveats and defensive pirouettes are wedged between all the words.

I think you and @thaumaturgy should chill a bit and put some dovetails on that C compiler. Oh and @thaumaturgy, I didn't say and don't think Python/Ruby are superior, my comment was addressing suitability for a task in the larger context. They were used as a concrete example. Please be a little more charitable or threads like these fall into a "someone is wrong on the internet".

I just realized that the people that probably need the most assistance arguing (in the classical sense) on the internet are the ones that make it. Ironic. Or, the barbarians will and have moved in.

*replying to your Also ...

Your interactive workflow sounds like (and this is only an indicative example) my use of IPython the console based advanced python repl.

# now here goes the philosophical rant portion of the reply

But at the same time, the host isn't bash it is Python, so I can also use the full power of the language without having to think about how to serialize text between tools I am composing.

Because composition is more important that each individual tool and the shell is lousy; awk/sed are shell enablers. If you are only working with line oriented text and some wellish defined delimiters, go for it.

We now operate over structured formats, we should have tools that compose well with those formats so that we an think about domain problems and not self-made problems.

Tools that have supplanted awk/sed are like jq and gron with new ones popping up all the time. This is where the Little Language innovation is going on. Something like jq is powerful enough both in expressivity and compute performance to part of my goto bag of tools.

https://github.com/stedolan/jq

https://github.com/TomNomNom/gron (it really makes json RIPgreppable)

https://github.com/BurntSushi/ripgrep

I hope the peanut gallery enjoys our exchange because we aren't even talking about the same things. Your building model sailboats and I am talking about problem solving efficiency so we can focus on domain problems.

I wish Erik was here to school us both.

Re: Useful sed scripts and patterns

#36

Post by sam_lowry_ »

baby wrote: post #32
Debug_Overload wrote: post #26

Earlier quoted context omitted.

How long does the relearning take exactly? My experience has been that if you've used these tools before, it's pretty easy to remember or refresh your memory. In that sense, they're not very different from other tools/syntaxes that you haven't used in a while. And if you do use them frequently enough, you might not even need to refresh/relearn anything.

I basically gave up on relearning them considering the number of times I’ve had to do it and the amount of effort involved to relearn them. Maybe it’s just me, but I just write python now.

You need to have a mental model for each tool and some idea of syntax associated with it. Once I learned about IFS,OFS, BEGIN, END and {}, as well as the fact that variables and their references have the same syntax, unlike in Perl... I could pick up coding in awk at any moment

Re: Useful sed scripts and patterns

#37

Post by [deleted] »

sitkack wrote: post #35
yesenadam wrote: post #33

Earlier quoted context omitted.

I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason. Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk . I got into sed a while ago but never use it. A…

Heh, Not over here doing arm chair yucking someone's yum, to you and everyone else in this shrub of the conversation. If you guys want to do SCA or model trains with ancient programming languages, go all in. I play around with Forth and APL as well. If the premise is the title of the post "day to day usage ..." then total task efficiency is important. Are you moving the goal posts and putting words in my mouth? If I…

[deleted]

Re: Useful sed scripts and patterns

#38

Post by agys »

f0e4c2f7 wrote: post #10
Waterluvian wrote: post #8

Something I crave in the worst way is a sensible story for accumulating a “toolbox” of snippets and scripts and commands. They don’t need to be ready to go, but ideally: - natural language searchable - add a small description - CLI to search, examine, and copy - not a plugin - sync with a GitHub repo.

One of the best hackers I ever worked with turned me onto this method. Use a flat text in the editor of your choice. Search as needed. Tags work because you can seach them. You delimit each section as a note. I do it like this --- Next note --- It seems limited and primative, you don't get any markup or fun stuff. But it's soooo good. There is nothing to break or update. Seach is the method I use most anyway in my ot…

I use .txt files for most of my tasks: all sort of lists, notes, snippets, even organized bookmarks (with comments). I use them also with some indentation and simple geometries for structured information (table-like or hierarchies).

Some of them are shared via DropBox and despite the risk of simultaneous edits we had basically no incidents: the simplicity and efficacy justifies it for us.

Re: Useful sed scripts and patterns

#39

Post by ivanhoe »

baby wrote: post #32
Debug_Overload wrote: post #26

Earlier quoted context omitted.

How long does the relearning take exactly? My experience has been that if you've used these tools before, it's pretty easy to remember or refresh your memory. In that sense, they're not very different from other tools/syntaxes that you haven't used in a while. And if you do use them frequently enough, you might not even need to refresh/relearn anything.

I basically gave up on relearning them considering the number of times I’ve had to do it and the amount of effort involved to relearn them. Maybe it’s just me, but I just write python now.

Makes sense if python is what you otherwise use for everyday work, it's always easiest if you can avoid context switching.

To me it's bash, sed and awk for the basic text-processing things, and then perl for anything that requires more programming. These are very similar syntaxes (and ways of thinking I'd say), so they fit together great - and also are close enough to php and js that I do at work to make the transition painless. But if I had to write it in python or ruby or God forbid C++ - which all I used a lot, but long time ago - I'm sure it would be a big struggle for me now.

Re: Useful sed scripts and patterns

#40

Post by ZeroGravitas »

baby wrote: post #22

I really dislike sed, awk, and all these utilities because I just can’t remember how to use them. Every time I want to use one of these tools I have to relearn the stuff, and then after a while I forget how to use it. If I don’t use it all the time I just forget. I really don’t think non-interactive CLIs are a good way to do complicated tasks.

I mostly find them useful when you can't be sure anything else will be installed.

What we really need is something that can "compile" down to sed for deployment in that kind of environment.

Then that compiler can contain all the weird stuff you need to remember.

Post reply on HN