Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bringing the Unix philosophy to the 21st century (2019)

blog.kellybrazil.com

Post reply on HN
Print view View original

131–140 of 151 posts

Re: Bringing the Unix philosophy to the 21st century (2019)

#131

Post by trasz »

jl6 wrote: post #30
c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

> what we'd really need is for all tool versions/variants output on all platforms (all GNU/Linux distros, all the BSDs, all embedded Linux variants, all commercial UNICES, etc.) to be the same, all the time The industry has approximated this state of affairs by approximating a GNU+Linux monoculture. Doesn’t matter that BSD tar behaves differently if ~nobody uses it.

Nobody except OSX, which means the great majority of Unix desktops use BSD tar.

Re: Bringing the Unix philosophy to the 21st century (2019)

#133

Post by kbrazil »

hjek wrote: post #119
kbrazil wrote: post #40

Earlier quoted context omitted.

Odd. What locale are you using? Should work fine with C or en_UTF8, per the readme.

$ echo $LANG en_DK.UTF-8 $ date 2021-08-23T08:07:02 CEST

Ah, yes - that locale is not supported. From the Caveats section:

For best results set the LANG locale environment variable to `C` or `en_US.UTF-8`. For example, either by setting directly on the command-line:

  $ LANG=C date | jc --date
It is possible to add support for more locales and you can always override the built in parsers with your own plugin to support it.

Re: Bringing the Unix philosophy to the 21st century (2019)

#134

Post by seewhat »

1MachineElf wrote: post #46
c0l0 wrote: post #19

So I've been writing shell scripts for about two decades, about 75% of that time professionally. Parsing the unstructured, text-based output of utilities is not the problem for anyone who's had maybe a few weeks of training. Most `... | grep ... | cut ... | sed ... | awk ...`-abominations the post laments can be replaced by a single informed call to `awk`, making everything a lot more elegant and concise. Having JSON…

About this few weeks training you mentioned for Unix-style parsing of unstructured text - what resources do you recommend?

I have occasionally dipped into this book for instruction and consider it a freely accessible reference:

Unix Text Processing, Dougherty and O'Reilly, 1987 https://www.oreilly.com/openbook/utp/

Cherry pick chapters like those on the Shell and AWK, and avoid those on troff, macros etc unless specifically interested.

Re: Bringing the Unix philosophy to the 21st century (2019)

#135

Post by HKH2 »

johnlorentzson wrote: post #125
quasarj wrote: post #48

Earlier quoted context omitted.

Mostly because S-expressions are terribly hard for humans to read... and everyone is already familiar with javascript syntax.

S-expressions aren't inherently more difficult to read than JSON, it's just a matter of getting used to it.

Chinese isn't more difficult to read than English; it's just a matter of getting used to it.

Re: Bringing the Unix philosophy to the 21st century (2019)

#136

Post by freshhawk »

woodruffw wrote: post #36
freshhawk wrote: post #26

Uh ... what about when I don't want to load the entire stream into memory before the next stage starts running? Are these implicit json arrays that stream out? Or do we now have incompatible json shell tools and streaming text tools as a permanent fixture? I'm excited about structured output ideas, but json? I'd much rather have streams of whitespace separated words than json. That's in that "No type system is better…

I agree with you in broad strokes, but as a piece of anecdata: I've had a lot of success building tools that emit and consume JSONL[1] instead of entire JSON documents. JSONL preserves the Unix pipeline's inherently parallel design (people tend to forget this, even when waxing about the Unix philosophy!) but gives us all of the nice typing of a JSON stream. That being said, I too will take a `sed` or `awk` one-liner…

Yeah, if we can't get something like edn (json but well designed and extensible) then jsonl is at least OK.

> people tend to forget this

There are people with strong opinions about this that don't even understand why you need something like JSONL in the first place and I'm surprised by that. If they never use unix pipelines I don't get why they feel we want to hear their hot take about how they should be redesigned.

Re: Bringing the Unix philosophy to the 21st century (2019)

#137

Post by kaba0 »

diegocg wrote: post #88
emodendroket wrote: post #80

Earlier quoted context omitted.

PowerShell is just so much nicer to use than anything where text munging is the only way to do things, and it’s just as “pluggable” as Unix shell commands. And it can output text (or JSON or XML or YAML) or whatever you want easily by piping into relevant commands (or just not piping anywhere if you wanted text). I don’t imagine those are going anywhere anytime soon but I think it is wrong to say the system cannot be…

I had to write a powershell script and my impression was that objects are a much worse interface than the worst text manipulation tricks. Because you are passing objects around it quickly becomes really hard to understand what is going on on the code and which object interfaces the script is using, because the object lives behind the pipe so to speak. It was my impression that it is too easy to create unreadable powe…

Powershell is slow to start up. Do you have any benchmark to show that it actually runs slower than reparsing the same data thousands of times as text?

Re: Bringing the Unix philosophy to the 21st century (2019)

#138

Post by hjek »

kbrazil wrote: post #133
hjek wrote: post #119

Earlier quoted context omitted.

$ echo $LANG en_DK.UTF-8 $ date 2021-08-23T08:07:02 CEST

Ah, yes - that locale is not supported. From the Caveats section: For best results set the LANG locale environment variable to `C` or `en_US.UTF-8`. For example, either by setting directly on the command-line: $ LANG=C date | jc --date It is possible to add support for more locales and you can always override the built in parsers with your own plugin to support it.

Yea, however this means that @ape4's point still stands:

> But, um, seems slightly kludgy to be obvious. What if the `ifconfig` format changes slightly.

Re: Bringing the Unix philosophy to the 21st century (2019)

#139

Post by cylinder714 »

eadmund wrote: post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Obligatory links:

Rivest's proposal, with source code: http://people.csail.mit.edu/rivest/sexp.html

McCarthy's Common Business Communication Language: http://jmc.stanford.edu/articles/cbcl.html

Re: Bringing the Unix philosophy to the 21st century (2019)

#140

Post by cylinder714 »

quasarj wrote: post #48
eadmund wrote: post #23

Why not go all the way and use a format capable of expressing code and data? I refer, of course, to S-expressions. They also have the benefit of properly handling numbers. Some might look at the absence of maps as a negative, but I think alists are preferable anyway due to their constant ordering.

Mostly because S-expressions are terribly hard for humans to read... and everyone is already familiar with javascript syntax.

Rivest's proposal includes an "advanced transport" representation, a more human-readable version. Here's a sample: http://people.csail.mit.edu/rivest/sexp-sample-a
Post reply on HN