Live data from Hacker News

The Rule of Silence (2006)

linfo.org

201–210 of 318 posts

Re: The Rule of Silence (2006)

#201
post #92
post #87

Earlier quoted context omitted.

Surely you can reconcile structured representations and something like the Unix command line. Imagine if the default wasn't bash, but something like Ruby + pipes (or some other terse language). What is the argument for shell scripts not working on typed objects? How much time has been lost, how many bugs have been created because every single interaction between shell scripts has to include its own parser. How many v…

> What is the argument for shell scripts not working on typed objects? How much time has been lost, how many bugs have been created because every single interaction between shell scripts has to include its own parser. How many versions of " get file created timestamp from ls" do we need? Aside: that's what the stat command is for. My big concern with types is how would you make sure that the output of a command will…

> Otherwise you'll have runtime type errors which would be just as bad as runtime parsing errors.

Parsing errors in script shells can easily go unnoticed (until you realize your data is corrupted).

Re: The Rule of Silence (2006)

#202
post #197
post #74

To play devil's advocate, part of the reason things like The Rule of Silence are talked about is because of the messy unix philosophy of treating everything like plain text. If structured data was embraced we would have developed appropriate tooling to interact with it in the way that we prefer. This runs very deep in unix and a lot of people are too "brainwashed" to think of other ways. Instead they develop other ex…

> ... the messy unix philosophy of treating everything like plain text. If structured data ... You are forgetting a crucial point: plain text is very well defined. Actually, it was already defined when the first Unix tools were being written. Using plain text means that you can use grep to search the logs of your program, even if your program was written yesterday and grep was written 40 years ago . Structured data?…

You would think plain text was well-defined, but along came Unicode and even that's not true anymore.

Re: The Rule of Silence (2006)

#203
post #116

I wish people would live by this rule more often.

Have you ever tried to talk to a person who gives absolutely no nonverbal cues as to whether or not they're listening to you? It's infuriating. Acks are important.

Yes I have. It's very common in some Eastern European countries, where the words stand on their own without embellishment. Jokes come with no intonation-based warnings. Similarily, listening is presumed and doesn't come with colorful Southern European–style cues.

It's not infuriating, it's just different.

Re: The Rule of Silence (2006)

#204
post #155
post #149

Earlier quoted context omitted.

> If structured data was embraced we would have developed appropriate tooling to interact with it in the way that we prefer. What kind of tooling might work with ad-hoc structured data and still getting all the tools to talk with each other like in Unix? How would it work without having to write input parsing rules, data processing rules, and output formatting/filtering/composing rules for each tool? I suspect that t…

>it's not very popular to pass around structured data There's an awful lot of JSON that gets passed around. That seems a reasonable compromise between readable text and some sort of structure.

JSON is an OK serialization format, and a terrible format for outputting human readable data. Let's take, for example, the output of 'ls -al' and imagine it were presented via JSON:

The keys 'mode', 'number of links', 'owner', 'group', 'size', 'last modified' would be repeated over and over again, stacked vertically.

Mode would remain an arbitrarily formatted string, or (worse) be broken into its own object for every attribute represented by the string.

A reasonably populated directory would fill multiple screens with cruft.

The formatting of the timestamp in the last modified field would still be an arbitrary string.

Comparing two files would require piping through another utility to extract just the appropriate fields and display them unadorned.

Sure, it might be moderately easier to consume in another program since you can simply iterate over a list and reference specific keys, but it's not really that hard to iterate over a list of lines and extract a specific field by number.

    ls -al | awk '{print $5,$9}'
vs.

    ls -al | jq '.[] | [.size,.name|tostring] | join(" ")'

Re: The Rule of Silence (2006)

#205
post #74

To play devil's advocate, part of the reason things like The Rule of Silence are talked about is because of the messy unix philosophy of treating everything like plain text. If structured data was embraced we would have developed appropriate tooling to interact with it in the way that we prefer. This runs very deep in unix and a lot of people are too "brainwashed" to think of other ways. Instead they develop other ex…

Seem to me that while shell scripts can be used by anyone given a bit of time, wysiwyg style, powershell is made by programmers for programmers.

Re: The Rule of Silence (2006)

#206
post #118

Earlier quoted context omitted.

Parsing structured text is slow and inefficient. Also reading out just one part of a data structure stored as text often requires either walking through the file character by character or first slurping the whole thing into memory.

...But when your system crashes, having all that data in an easily accessible manner (regardless of what tools you have on hand) is a major win.

Let's not forget that when your system crashes, all of these easy-to-read text files are actually stored in a binary format, sometimes scattered in pieces, and require special tools to extract.

Re: The Rule of Silence (2006)

#207
post #106
post #15

Earlier quoted context omitted.

> Note that the "rule of silence" (combined with the habit of writing documentation like longform essays) is also one factor that makes unix-like systems newbie-unfriendly. (Famous example: trying to exit vi) $ man foo *scroll to the end with the EXAMPLES section* There should be an option for that. man --take-me-to-the-examples foo

If you want a fast way to read the EXAMPLES section only for a command, here is a shell function which creates an ‘eg’ command which only displays the “EXAMPLES” section of manual pages: eg(){ MAN_KEEP_FORMATTING=1 man "$@" 2>/dev/null \ | sed --quiet --expression='/^E\(\x08.\)X\(\x08.\)\?A\(\x08.\)\?M\(\x08.\)\?P\(\x08.\)\?L\(\x08.\)\?E/{:a;p;n;/^[^ ]/q;ba}' \ | ${MANPAGER:-${PAGER:-pager -s}} } Usage: $ eg tar EXAM…

Here's mine:

  examples ()
  {
    man $1 | less +/^EXAMPLES
  }
Usage:

  $ examples su
  EXAMPLES
       su -m man -c catman
              Starts a shell as user man, and runs the command catman.  You will
              be asked for man's password unless your real UID is 0.  Note that
              the -m option is required since user “man” does not have a valid
              shell by default.  In this example, -c is passed to the shell of
              the user “man”, and is not interpreted as an argument to su.
       su -m man -c 'catman /usr/share/man /usr/local/man'
              Same as above, but the target command consists of more than a
              single word and hence is quoted for use with the -c option being
              passed to the shell.  (Most shells expect the argument to -c to be
              a single word).
       su -m -c staff man -c 'catman /usr/share/man /usr/local/man'
              Same as above, but the target command is run with the resource
              limits of the login class “staff”.  Note: in this example, the
              first -c option applies to su while the second is an argument to
              the shell being invoked.
       su -l foo
              Simulate a login for user foo.
       su - foo
              Same as above.
       su -   Simulate a login for root.

Re: The Rule of Silence (2006)

#209

Earlier quoted context omitted.

Structured text is good. Very good, in fact. It might even be idea. structured binary data , less so, at least as a storage format. I want to be able to look at your file format using tools that haven't been specialized to the task. Is that so wrong?

Well, I'm advocating for structured text , not binary . Mostly because I haven't seen a future-proof binary format yet, and editing binary formats indeed would require special tooling. I think - for a data exchange protocol meant to be used between many applications - going structured text instead of binary is a worthwhile tradeoff of little lower efficiency vs. much better accessibility. EDIT: Some comments here are…

Let's say I want to know how many days since an asset on my web server has been modified. With bash + some standard unix tools, from the top of my head I have to do something like this:

    curl -svo /dev/null http://example.com/file 2>&1 | grep Last-Modified | cut -d ' ' -f 3-
And that's just to get the last modified date in text form. Now I'm writing a script that parses that date and gets today's date, convert them to days, and subtract. YUCK!

Wouldn't it be nice if your shell could do this?

    curl(http://example.com/file).response_headers.Last-Modified.subtract(date().now).days
I think it would be nice.

Re: The Rule of Silence (2006)

#210
post #130

Earlier quoted context omitted.

That's my primary issue with UNIX culture. It took a huge step backwards by deciding to work with unstructured text. It wasn't a wrong turn , mind you. It was backtracking on known and understood best practices all the way and then picking a wrong turn. And only now people seem to rediscover what was in common use in the era before UNIX - the virtues of structured text. I guess our industry is meant to run in circles…

It's not like unstructured piped text is the only possible way to work. It's widely used precisely because it's so expedient. If you use structured data, then every program in the sequence has to understand the structure. If you just smash everything flat into a stream of text, you can then massage it into whatever form you need. It's not always the best way to approach a problem, but it's not meant to be. It's duct…

The way i see it, shell scripting and piping allows non-programmers to get their feet wet, one command at a time.

You run a command, look at the output, now you know what the next command in the pipeline will see, and can add adjustments as needed.

Powershell etc seems to be more programmer oriented in that one keep thinking in terms of variables and structures that gets passed around.

And this seems to be the curse of recent years. More and more code is written by programmers for programmers. Likely because everyone has their head in the cloud farms and only the "front end" people has to deal with the users.

UNIX came when you risked having admins and users sitting terminal to terminal in the same university room. Thus things got made that allowed said users to deal with things on their own without having to nag the admins all the time.

Post reply on HN