This is cool for English text. But once you get Unicode with various ways to represent é, whew lad. This get shitty quickly in the shell.
Just drop the eighth bit and use 7-bit ASCII like God intended and it's not a problem... :-)
Text Processing in the Shell
81–90 of 109 posts
Re: Text Processing in the Shell
#82Sometimes I find it strange - in both good and bad way - that we are, in 2020, learning tools and languages designed and build in the 80s, with models and constraints of the time, with 40 years of layers of backwards compatibility, and actually going sometimes back to the 70s I am still learning tools designed around the constraints of teleprinters Sure, it’s the same on Windows side (and macOS side with their classi…
The English language is several hundreds of years old, and the ADN is several hundreds of millions years old. Everything we are based on is legacy; I do not see why it should be different in computing.
What do you mean by ADN? Possibly a non-English equivalent for DNA? (My guess based on EN Wikipedia's disambiguation page.)
Re: Text Processing in the Shell
#83I'll preface my question by saying that I'm not a dev. Why do this kind of work in the shell? Isn't it better to do this in a programming language that can run on all operating systems? What are Windows users supposed to do?
As an also-not-a-dev:
Speed. Portability. Muscle memory. I've spent ten years troubleshooting UNIX applications, so most of these commands are fairly well-ingrained into my mode of thinking when I have data that I've got to parse.
To boot, these shell utilities were written by people way smarter than me. I have far more confidence that they will handle edge cases in the data stream infinitely better than whatever dinky little Python script I might try to hash out.
Re: Text Processing in the Shell
#84Earlier quoted context omitted.
The English language is several hundreds of years old, and the ADN is several hundreds of millions years old. Everything we are based on is legacy; I do not see why it should be different in computing.
What do you mean by ADN? Possibly a non-English equivalent for DNA? (My guess based on EN Wikipedia's disambiguation page.)
Yep, sorry, I'm used to this acronym in french.
Re: Text Processing in the Shell
#85A little suggestion for the authors, they mentioned xargs, I think [GNU parallel]( https://www.gnu.org/software/parallel/ ) might work a mention too, since it is a kind of modern successor that can use many computers to run tasks.
This you have to install, xargs is everywhere. Also with the -P flag you can parallelize the most common cases.
Personally I like parallel better because of the `--bar` option and `{}`, `{.}`, `{/}` and `{/.}`. And I usually just use it with `-P 1` anyway.
Re: Text Processing in the Shell
#86Lately I've started to use Perl instead of sed for replacing text. Its regex support is much better IMO:
cat a.txt | perl -pe 's/banana-(\d)/papaya-$1/g'
perl -i -pe 's/banana-(\d)/papaya-$1/g' a.txtRe: Text Processing in the Shell
#87Earlier quoted context omitted.
not all data comes from sources that you control and have chosen how to output.
https://github.com/kellyjonbrazil/jc can come pretty handy there
oh my god, why?! just why!?
What the world needs is the inverse program of "jc", where an unparseable json string is expanded into a flat list of lines all of the form "field.subfield=value"
Re: Text Processing in the Shell
#88Earlier quoted context omitted.
https://github.com/kellyjonbrazil/jc can come pretty handy there
oh my god, why?! just why!? What the world needs is the inverse program of "jc", where an unparseable json string is expanded into a flat list of lines all of the form "field.subfield=value"
ok well I can understand not wanting json when all you need is something simpler, but I'm not sure if I understand unparseable - I mean if it is JSON then it is parseable.
Re: Text Processing in the Shell
#89I've always viewed `awk '!a[$0]++'` as superior to `sort | uniq` because it preserves order and does not have to sort the data first before deduplicating. But `sort | uniq` is much easier to remember.
It appears sort has a --stable flag for order preservation. I actually thought sort was stable by default already, but now I know.
Re: Text Processing in the Shell
#90Earlier quoted context omitted.
... and pretty much all of those were addressed, outwith Unix, by the evolution of the 1960s terminal I/O model into the console I/O model during the 1980s. You even forgot to mention one of the things that was addressed: input. Terminal I/O input, done properly, requires a full ECMA-48 decoder state machine, with bodges to accommodate non-conformant warts from the Linux KVT, SCO Console, and RXVT. This is all too of…
> the evolution of the 1960s terminal I/O model into the console I/O model during the 1980s. Are you talking about MS-DOS-style memory-diddling to achieve things like colors and reverse video?
Read https://news.ycombinator.com/item?id=17238350 and https://news.ycombinator.com/item?id=22603878 in this very discussion.