Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Text Processing in the Shell

blog.balthazar-rouberol.com

Post reply on HN
Print view View original

61–70 of 109 posts

Re: Text Processing in the Shell

#61

Post by Koshkin »

Timpy wrote: post #45
ekianjo wrote: post #38

Earlier quoted context omitted.

> I am still learning tools designed around the constraints of teleprinters We are still reading books using a 2k year old alphabet to represent ideas. Not sure why it would be surprising that text manipulation is still the norm pretty much in everything we do, including computing.

If we're going to update, I think we should stop using C for /s/ and /k/ noises (we already have characters for those). It can exclusively represent /tʃ/, as in "China" would become "Cina". We don't need X either for much the same reason, which is all the better because we'll need a character for /ʃ/. "Delicious" will become "delixious", "shadow" will become "xadow", "nation" will become "naxion". I know this sounds…

This has been done before: http://www.davidpbrown.co.uk/jokes/european-commission.html.

Re: Text Processing in the Shell

#62

Post by Koshkin »

dejj wrote: post #40
seemslegit wrote: post #2

Just Don't. Unix-style text stream processing was super cool in 80s-90s but is born tech-debt today.

I love to cook 5-course meals for friends and family. But sometimes when I'm all alone by myself, I slice up a bun and slap meat on it. It's as fast and tastes just as good as in the 80ies.

Sure, but try to eat a juicy sandwich that was made in the 80s.

Re: Text Processing in the Shell

#63

Post by ken »

Timpy wrote: post #45
ekianjo wrote: post #38

Earlier quoted context omitted.

> I am still learning tools designed around the constraints of teleprinters We are still reading books using a 2k year old alphabet to represent ideas. Not sure why it would be surprising that text manipulation is still the norm pretty much in everything we do, including computing.

If we're going to update, I think we should stop using C for /s/ and /k/ noises (we already have characters for those). It can exclusively represent /tʃ/, as in "China" would become "Cina". We don't need X either for much the same reason, which is all the better because we'll need a character for /ʃ/. "Delicious" will become "delixious", "shadow" will become "xadow", "nation" will become "naxion". I know this sounds…

Turkish did this right, IMHO, when it adopted a Latin alphabet. No diphthongs at all. One letter, one sound. The “sh“ sound is spelled ş, and “ch” is ç.

You also need to add more vowels, as there are way more than 5 vowel sounds, but that’s not hard. Umlauts are already common in many languages.

Re: Text Processing in the Shell

#64

Post by asicsp »

clircle wrote: post #60

I'll preface my question by saying that I'm not a dev. Why do this kind of work in the shell? Isn't it better to do this in a programming language that can run on all operating systems? What are Windows users supposed to do?

Well, there are ways to use such commands on windows too, like WSL, git-bash, cygwin, etc. And, there's powershell on windows (though I haven't used it and not sure about its capabilities)

As to why use shell, that depends on your use case and working environment. Shell is something like an IDE [0] where you can solve multiple tasks from single environment. You don't have to use multiple programs (window manager, text editor, IDE, etc). Since it is all text, you can save and repeat a command, share it with others, edit a previously written command, etc. This is quite different from a GUI based workflow. Personally, I find using command line more productive, but as mentioned earlier, it'll depend on the task at hand.

[0] https://sanctum.geek.nz/arabesque/series/unix-as-ide/

Re: Text Processing in the Shell

#66

Post by superasn »

latenightcoding wrote: post #47

OP you should mention perl one-liners in upcoming chapters https://catonmat.net/introduction-to-perl-one-liners

Totally agree. I always use Perl instead of sed because sed doesn't support PCRE and it's far easier to write something like '\d+' instead of [:digit:] or whatever the sed equivalents are.

Re: Text Processing in the Shell

#67

Post by vlovich123 »

clircle wrote: post #60

I'll preface my question by saying that I'm not a dev. Why do this kind of work in the shell? Isn't it better to do this in a programming language that can run on all operating systems? What are Windows users supposed to do?

> Why do this kind of work in the shell

It can be quicker & easier to write. Also individually these tools can outperform any code you write by hand. Shells are also common on a far wider range of operating systems than typical programming languages like Python et all. Additionally if you want to improve your productivity you may create your own shortcuts (e.g. "build_myproject" which understands what that entails & may involve some amount of text processing among other things). It's typically far more convenient (shorter, simpler & generally easier to understand) to invoke other programs from shell languages since that's what their programming interface is optimized around.

Sometimes it's good to even wrap the entrypoint for common scripting languages like Python in shell so that, for example, you can setup a virtual environment to run out of or use the proper version of Python.

> Isn't it better to do this in a programming language that can run on all operating systems?

Bash & coreutils have been ported to every operating system under the sun (including Sun operating systems). They're even more common & available than any other programming language that doesn't require a compiler (e.g. `adb shell` will get you into an environment where you can grep & do these operations even though there's generally no python or other scripting language available).

> What are Windows users supposed to do?

* cygwin

* WSL

* Windows ports of Bash (http://win-bash.sourceforge.net/, https://gitforwindows.org/, etc)

* msys

Let me conclude this post that you shouldn't really write anything complex or maintained by multiple people in shell if you can avoid it and if you can make guarantees about (for example) the available of a Python interpreter. That doesn't mean that shell scripts aren't a valuable and important part of the development ecosystem.

Re: Text Processing in the Shell

#68

Post by petepete »

shp0ngle wrote: post #12

Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s I am still learning tools designed around the constraints of teleprinters Sure, it’s the same on Windows side (and macOS side with their classi…

The beauty of it is that these old tools play extremely nicely with their modern counterparts. ag, fzf, fd and tmux are key tools I use to aid my work on a daily basis alongside many of the classics listed in this article.

Re: Text Processing in the Shell

#69

Post by gjm11 »

ken wrote: post #58
enriquto wrote: post #21

Earlier quoted context omitted.

The English language is several hundreds of years old, and the ADN is several hundreds of millions years old. Everything we are based on is legacy; I do not see why it should be different in computing.

And English is terrible as a language! We’ve been making fun of how bad it is for centuries. Compare languages like Turkish or Korean, which are still natural languages but got a well-thought-out tune-up in the not-so-distant past. Why can’t my language have a pluralization rule (for example) that’s so simple and regular it takes 1 minute to teach, and 3 minutes to master? Or an alphabet that looks like how it is pro…

It could! Given enough international cooperation, we could all adopt a lightly modified version of English that's more regular and easier to teach and learn.

And then, over the coming years, it would absorb words from other languages with different rules, and evolve according to what people find easy or convenient (or just at random), and in a little while it would be irregular once again.

As you said, Turkish and Korean were changed recently. Give them time and some of that regularity will get chipped away.

(Also, of course: make substantial changes to how the language works and suddenly no one can comfortably read any of the vast quantities of existing writing unless it's translated. And some of that existing writing is really good.)

Re: Text Processing in the Shell

#70

Post by 0xdeadbeefbabe »

shp0ngle wrote: post #12

Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s I am still learning tools designed around the constraints of teleprinters Sure, it’s the same on Windows side (and macOS side with their classi…

Sure it's hard for us, but what about creatures that have 28 eyes and 14 brains? In other words, we also have more CPU cores now. I'm not sure I'm keeping all my CPU cores happy.

And no I don't think it's just a problem for the OS. Although, that's probably a popular idea.

Post reply on HN