Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Text Processing in the Shell

blog.balthazar-rouberol.com

Post reply on HN
Print view View original

11–20 of 109 posts

Re: Text Processing in the Shell

#12

Post by shp0ngle »

Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s

I am still learning tools designed around the constraints of teleprinters

Sure, it’s the same on Windows side (and macOS side with their classic OS compatibility layers still present, like all the HFS stuff). Not bashing bash here.

Surely our computers have very different models of operation than PDP-11, yet we are sometimes pretending it doesn’t

Re: Text Processing in the Shell

#13

Post by pletnes »

RMPR wrote: post #6

A little suggestion for the authors, they mentioned xargs, I think [GNU parallel]( https://www.gnu.org/software/parallel/ ) might work a mention too, since it is a kind of modern successor that can use many computers to run tasks.

This you have to install, xargs is everywhere. Also with the -P flag you can parallelize the most common cases.

Re: Text Processing in the Shell

#14

Post by SPBS »

I've always viewed `awk '!a[$0]++'` as superior to `sort | uniq` because it preserves order and does not have to sort the data first before deduplicating. But `sort | uniq` is much easier to remember.

Re: Text Processing in the Shell

#15

Post by throwaway_pdp09 »

shp0ngle wrote: post #12

Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s I am still learning tools designed around the constraints of teleprinters Sure, it’s the same on Windows side (and macOS side with their classi…

I find these posts annoying that say we're anchored in past tech and we must be able to do better, but never suggest how.

Please make a suggestion or two.

(NB ultimately our character handling is based on 'writing' which goes back thousands of years, not 50, and it survives well).

Re: Text Processing in the Shell

#17

Post by Lio »

shp0ngle wrote: post #12

Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s I am still learning tools designed around the constraints of teleprinters Sure, it’s the same on Windows side (and macOS side with their classi…

It’s because fundamentally, when you get right down to it, we’re manipulating text files.

Text is a very dense way to represent logic and ideas.

This is not like legacy software that you can rewrite.

You can add a GUI based on ideas from the late 70s/80s if you like but you’re still unlikely to come up with a more succinct way to represent logic than can be held in a text file.

So it follows that small tools that deal directly with manipulating text will be useful as long as text is useful.

Re: Text Processing in the Shell

#19

Post by asicsp »

SPBS wrote: post #14

I've always viewed `awk '!a[$0]++'` as superior to `sort | uniq` because it preserves order and does not have to sort the data first before deduplicating. But `sort | uniq` is much easier to remember.

Or just `sort -u` (if you are using GNU sort, not sure about other implementations)

Another difference is that sort is optimized to handle large files [0]

[0] https://unix.stackexchange.com/questions/279096/scalability-...

Post reply on HN