Live data from Hacker News

Bringing the Unix philosophy to the 21st century (2019)

blog.kellybrazil.com

141–150 of 151 posts

Re: Bringing the Unix philosophy to the 21st century (2019)

#141
post #138

Earlier quoted context omitted.

Ah, yes - that locale is not supported. From the Caveats section: For best results set the LANG locale environment variable to `C` or `en_US.UTF-8`. For example, either by setting directly on the command-line: $ LANG=C date | jc --date It is possible to add support for more locales and you can always override the built in parsers with your own plugin to support it.

Yea, however this means that @ape4's point still stands: > But, um, seems slightly kludgy to be obvious. What if the `ifconfig` format changes slightly.

Absolutely! I didn't write JC to be the be-all end-all. I wrote it because I believe using structured text between processes is usually better than plain text and this tool allows people to try it out and see for themselves. The goal is for people to see the benefits and require tools to output structured data, either for old utilities, or especially new ones.

Lots of people have asked for this, but the argument has always been that it can't be done, it's too hard. So I created JC to help open minds and change behavior. As I've said many times: the goal of JC is for JC to never have to exist. Hopefully it will persuade people that there is a better way and we should expect better from our existing tools without having to completely change the way we do things today.

JC supports over 70 programs and file-types today. I slowly, incrementally added more and more parsers over the last two years. Now it's to the point that it's hard to find popular apps that don't have coverage. lately, I get requests to add parsers for apps that already provide JSON output, which I don't do. :)

And those old apps like `ifconfig`? Many of those haven't been touched in a decade. There's not a huge risk of the output changing any time soon. (Believe me, I know - I went through the source code in several of these utilities to be able to figure out what to call some of the undocumented fields)

What I'm trying to say is that there are 100 reasons to say it won't work until you actually try it and you find that it actually works pretty well and opens up possibilities you hadn't thought of before.

Re: Bringing the Unix philosophy to the 21st century (2019)

#142
post #131
post #30

Earlier quoted context omitted.

> what we'd really need is for all tool versions/variants output on all platforms (all GNU/Linux distros, all the BSDs, all embedded Linux variants, all commercial UNICES, etc.) to be the same, all the time The industry has approximated this state of affairs by approximating a GNU+Linux monoculture. Doesn’t matter that BSD tar behaves differently if ~nobody uses it.

Nobody except OSX, which means the great majority of Unix desktops use BSD tar.

HP-UX, AIX and Solaris are still around.

Plus there are a couple of POSIX like OSes for embedded deployment.

Re: Bringing the Unix philosophy to the 21st century (2019)

#143
post #94

Earlier quoted context omitted.

You parse the s-exprs and execute them in the context of a namespace of data constructors. Then you can have whatever data structures in memory that are defined by the constructors. This is NOT equivalent to having one data structure to cover both as Lua does. It is having one text format that can construct any kind of data structure in memory for which you have constructors defined.

Okay, but we’re talking about a lingua franca for exchanging plain old data between different programs. You would have to pick your data constructors in advance. I’m saying that arrays and maps give good bang for the buck, so you don’t really need to define anything beyond those. And if you accept that, having special syntax for arrays and maps is more convenient and readable than S-exprs.

> And if you accept that, having special syntax for arrays and maps is more convenient and readable than S-exprs.

S-expressions are arrays, and maps are really just degenerate unsorted arrays of key-value pairs. Taking a look at https://json.org/example.html, I think this is easily more readable:

    (menu
      (id file)
      (value File)
      (popup
        (item (value New) (onclick "CreateNewDoc()"))
        (item (value Open) (onclick "OpenDoc"))
        (item (value Close) (onclick "CloseDoc()"))))
than:

    {"menu": {
      "id": "file",
      "value": "File",
      "popup": {
        "menuitem": [
          {"value": "New", "onclick": "CreateNewDoc()"},
          {"value": "Open", "onclick": "OpenDoc()"},
          {"value": "Close", "onclick": "CloseDoc()"}
        ]
      }
    }}

Re: Bringing the Unix philosophy to the 21st century (2019)

#144

Earlier quoted context omitted.

What, besides the actual result, do you write to stout? (And shouldn't it go to stderr instead?)

Typically informational messages/log message. Ideally yes logging should go to stderr but some applications that intercept the file descriptors consider anything to stderr as error and it is not always possible to change the handling for those applications. E.g. Octopus Deploy[0] does this: it doesn't outright fail the step assuming other "successful" steps happen after that but if the last command writes to stderr,…

I guess I just wanted to be on a high horse and declare that anybody who puts something other than what you asked for on stdout is doing it WRONG. But you're right, fixing the universe to fit the convention isn't always in the cards.

If I could design it all from scratch I'd have a convention for getting a program to tell me:

- info, user prompts, debug -> stderr

- foo data, bar data -> stdout

That way I could use those defaults sometimes, but other times specify per-execution preferences that would route the data differently.

Re: Bringing the Unix philosophy to the 21st century (2019)

#145

Earlier quoted context omitted.

Typically informational messages/log message. Ideally yes logging should go to stderr but some applications that intercept the file descriptors consider anything to stderr as error and it is not always possible to change the handling for those applications. E.g. Octopus Deploy[0] does this: it doesn't outright fail the step assuming other "successful" steps happen after that but if the last command writes to stderr,…

I guess I just wanted to be on a high horse and declare that anybody who puts something other than what you asked for on stdout is doing it WRONG. But you're right, fixing the universe to fit the convention isn't always in the cards. If I could design it all from scratch I'd have a convention for getting a program to tell me: - info, user prompts, debug -> stderr - foo data, bar data -> stdout That way I could use th…

I was looking more into this and found something interesting[0]:

> A variable can be assigned the nameref attribute using the -n option to the declare or local builtin commands (see Bash Builtins) to create a nameref, or a reference to another variable. This allows variables to be manipulated indirectly. Whenever the nameref variable is referenced, assigned to, unset, or has its attributes modified (other than using or changing the nameref attribute itself), the operation is actually performed on the variable specified by the nameref variable’s value. A nameref is commonly used within shell functions to refer to a variable whose name is passed as an argument to the function.

So something like below:

    set_var() {
        local -n vname=$1   # use nameref for indirection
        # some processing and assign value
        vname="abcd"
    }

    use_var() {
        local output
        set_var output # call function to populate the variable
        echo "output=$output"
        unset -n output
    }

    use_var

I guess this is pretty close to "clean". Unfortunately, it is only available in bash 4.3, if I'm not wrong, so I don't have a choice to use it (our hosts use 4.2.x).

[0]: https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

Re: Bringing the Unix philosophy to the 21st century (2019)

#146
why in the world would you pipe through grep to select certain lines and then use awk? You could just select lines with awk. The same could be said of cut. If you want to use grep and cut that's fine, but using grep and cut with awk implies you are doing it wrong.

Cut for field 1 (`cut -d/` -f1) is a cheesy way of saying we want to delimit fields with '/' too. So add that to awk via script variable or via -F command-line option.

Re: Bringing the Unix philosophy to the 21st century (2019)

#147
post #67
post #63

Earlier quoted context omitted.

But json already has quotes, commas, brackets and braces :-)

Yeah, apparently finding misplaced quotes, commas, brackets and braces is magically easier than misplaced parentheses. :)

You jest, but I find it easier to visually parse code organized using parentheses, quotes, commas, and brackets compared to code organized using parentheses, parentheses, parentheses, and parentheses. The latter approach is simple and elegant but makes it hard for me to read someone else's code.

I think the first reason is that the distinct characters provide a kind of visual checksum which makes me (slightly) more confident when initially matching a closing character to the correct starting character.

The second reason is that each character has a conventional meaning which makes it possible to form an initial guess as to its purpose.

Having said this, I freely admit this opinion may be colored by my only (paid) experience writing Lisp which was for a sprawling 20 year-old AI codebase mostly written by professors and graduate students who were often learning Lisp as they went.

Re: Bringing the Unix philosophy to the 21st century (2019)

#148
post #146

why in the world would you pipe through grep to select certain lines and then use awk? You could just select lines with awk. The same could be said of cut. If you want to use grep and cut that's fine, but using grep and cut with awk implies you are doing it wrong . Cut for field 1 (`cut -d/` -f1) is a cheesy way of saying we want to delimit fields with '/' too. So add that to awk via script variable or via -F command…

It's probably not the most correct, but it is a very common practice[0].

[0]https://stackoverflow.com/questions/23934425/parse-ifconfig-...

Re: Bringing the Unix philosophy to the 21st century (2019)

#149

Earlier quoted context omitted.

Are you using one of these exclusively? Which one have you chosen and why?

I use murex exclusively. It’s still beta quality so not totally bug free but the bugs are rarely showstoppers and it’s still in active development so fixes are usually just one GitHub issue away. I don’t use it on servers, I stick with Bash for that. But I do use it as my primary local shell. As for why murex, that’s a combination of personal preference and simply not being aware of other shells until after I’d alrea…

Thank you, I'll check out Murex.

Re: Bringing the Unix philosophy to the 21st century (2019)

#150

Earlier quoted context omitted.

Okay, but we’re talking about a lingua franca for exchanging plain old data between different programs. You would have to pick your data constructors in advance. I’m saying that arrays and maps give good bang for the buck, so you don’t really need to define anything beyond those. And if you accept that, having special syntax for arrays and maps is more convenient and readable than S-exprs.

> And if you accept that, having special syntax for arrays and maps is more convenient and readable than S-exprs. S-expressions are arrays, and maps are really just degenerate unsorted arrays of key-value pairs. Taking a look at https://json.org/example.html , I think this is easily more readable: (menu (id file) (value File) (popup (item (value New) (onclick "CreateNewDoc()")) (item (value Open) (onclick "OpenDoc"))…

Each to their own!

A big part of the difference is that all the JSON keys are quoted, which I agree is ugly (I like JSON5 myself).

You’ve also omitted the “menuitem” from the S-expr version. That could have been omitted from the JSON too but I assume it’s meant to be there for some good reason.

Post reply on HN