Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CLI text processing with GNU awk

learnbyexample.github.io

Post reply on HN
Print view View original

21–30 of 136 posts

Re: CLI text processing with GNU awk

#21

Post by swores »

dotancohen wrote: post #15
thangngoc89 wrote: post #5

I have been using ChatGPT for generating these kind of small CLI like this. My prompts look like this: - use jq to count a nested array "a.b.c.d" - find and delete empty folders using `find` - find and replace text using sed/awk I found that using ChatGPT for these purposes boosted my productivity tremendously.

ChatGPT is a great time saver for those who already know how to use awk. But it should not be used by those who are unfamiliar. Just an example, I saw someone come up with a great awk line to change some text in a nested directory. He then pasted into bash. Only once the server went down did anybody realize that he forgot to cd into the proper directory and he wiped out not only the server config but also all the use…

That's not really a ChatGPT issue, people pasting in slightly wrong commands (or right commands in the wrong folder) is a tale as old as time - well, as old as linux at least. Short of saying that nobody who's already an expert should ever touch a CLI, the lesson from that story is "be as careful as possible, then be more careful, and also have backups of everything" not "don't use a LLM to help".

Re: CLI text processing with GNU awk

#22

Post by swores »

delta_p_delta_x wrote: post #14
qalmakka wrote: post #13

Awk is fine and dandy but, like wity Sed, I think that it's almost always replaceable with Perl which is way nicer to use, and ubiquitous. Every OS (except Windows) I laid my hands on in the last 15 years has had Perl installed in either its default install or pulled in as a dependency almost immediately (a LOT of stuff depends on Perl in any Unix system). This is, unless you are running on an embedded environment, b…

> except Windows On Windows, you use PowerShell.

They were talking about whether or not the OS comes with Perl by default, not whether it has a CLI at all.

Re: CLI text processing with GNU awk

#23

Post by swores »

I suspect that anyone reading this thread is likely to be equally interested in "Ask HN: Share a shell script you like" from a fortnight ago (though at 78 comments, it didn't get as much traction / comments as I hoped it would when I saw it)

https://news.ycombinator.com/item?id=37112991

Re: CLI text processing with GNU awk

#25

Post by empthought »

qalmakka wrote: post #13

Awk is fine and dandy but, like wity Sed, I think that it's almost always replaceable with Perl which is way nicer to use, and ubiquitous. Every OS (except Windows) I laid my hands on in the last 15 years has had Perl installed in either its default install or pulled in as a dependency almost immediately (a LOT of stuff depends on Perl in any Unix system). This is, unless you are running on an embedded environment, b…

Perl is not nicer to use.

Re: CLI text processing with GNU awk

#26

Post by asicsp »

swores wrote: post #20
asicsp wrote: post #3

Hello! Author here. I am pleased to announce a new version of my "CLI text processing with GNU awk" ebook. Learn the `GNU awk` command step-by-step from beginner to advanced levels with hundreds of examples and exercises. This book will dive deep into field processing, show examples for filtering features, multiple file processing, how to construct solutions that depend on multiple records, how to compare records and…

I'm curious to know how many people you get paying for something like your "Magical one-liners", and also whether you've ever experimented with a "choose to pay after" model? I ask because it's the kind of thing that I can imagine finding useful enough to pay $5 (or $15) for, but I can also imagine it being something that contains nothing I don't already have saved in my personal "one liners" file, so I'm not really…

I use Gumroad/Leanpub to sell my ebooks. As far as I know, they don't support the "choose to pay after" model.

You can see the number of paid sales for the bundles under the "I want this!" button. When the price is 0, it shows the total of both paid/free users.

I started selling ebooks about 5 years back. Where I live, my monthly living cost is just $150. While the first two years of sales were just about enough to cover my costs, the last three years have been much better - I can continue being self-employed :)

Re: CLI text processing with GNU awk

#27

Post by asicsp »

qalmakka wrote: post #13

Awk is fine and dandy but, like wity Sed, I think that it's almost always replaceable with Perl which is way nicer to use, and ubiquitous. Every OS (except Windows) I laid my hands on in the last 15 years has had Perl installed in either its default install or pulled in as a dependency almost immediately (a LOT of stuff depends on Perl in any Unix system). This is, unless you are running on an embedded environment, b…

I wouldn't put Perl as easier to use, but it certainly is more powerful and has a vast ecosystem. And it is more portable, since there's no need to worry about GNU/BSD/etc variations.

And I wrote a book for Perl one-liners as well (https://learnbyexample.github.io/learn_perl_oneliners/), which I'm currently revising (like I did for the grep/sed/awk ebooks).

Re: CLI text processing with GNU awk

#28

Post by asicsp »

swores wrote: post #23

I suspect that anyone reading this thread is likely to be equally interested in "Ask HN: Share a shell script you like" from a fortnight ago (though at 78 comments, it didn't get as much traction / comments as I hoped it would when I saw it) https://news.ycombinator.com/item?id=37112991

There was a similar discussion 5 months back: https://news.ycombinator.com/item?id=35122780 (332 points | 328 comments)

And here's one from last year: https://news.ycombinator.com/item?id=32467957 (374 points | 294 comments)

Re: CLI text processing with GNU awk

#29

Post by swores »

asicsp wrote: post #28
swores wrote: post #23

I suspect that anyone reading this thread is likely to be equally interested in "Ask HN: Share a shell script you like" from a fortnight ago (though at 78 comments, it didn't get as much traction / comments as I hoped it would when I saw it) https://news.ycombinator.com/item?id=37112991

There was a similar discussion 5 months back: https://news.ycombinator.com/item?id=35122780 (332 points | 328 comments) And here's one from last year: https://news.ycombinator.com/item?id=32467957 (374 points | 294 comments)

Thanks :)

Re: CLI text processing with GNU awk

#30

Post by chasil »

ymgch wrote: post #7

What is better? Starting with awk or sed?

A sed binary is usually much smaller than an awk binary, either POSIX or GNU. The memory footprint of sed will be much more compact.

However, sed has grown out of the command language used by the tty editors, and is more difficult to program (although it is Turing-complete).

The awk language implements much of the syntax of C, and it is not difficult to write a very slow and inefficient script. This inefficiency is harder to reach in sed, because it takes more effort to abuse it.

O'Reilly's book on sed and awk is available free online, both to browse and to download as a ZIP.

https://docstore.mik.ua/orelly/unix/index.htm

Post reply on HN