91–100 of 112 posts
His example is poor, but the message is that the body of a record can cross line boundaries. While the UNIX tool chain is predicated on the concept line == record, this doesn't have to be the case. With a generic record level marshalling system the class of problems solved by composing command line tools together would be greatly expanded. What Pike describes is analogous to the RecordReader in Hadoop.
Earlier quoted context omitted.
When I talk about a "representation," I mean an in-memory format. For example, the "representation" of an HTML tree is the DOM. Yes, you have to agree on a serialization format (JSON, Protocol Buffers, etc), but that's not the same thing. From a serialization format you can represent the data however you see fit in your process. For example, a C++ user might represent a string as a std::string object whereas a Python…
Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?
Earlier quoted context omitted.
>>Let me ask you this; how would you do the equivalent of this hypothetical command? >> $ ls | structured-grep 'file.size > 1M' # find . -maxdepth 1 -size +1M really, flat text is fine. Maybe it's not perfect, but it's good enough that most want to not add any complexity to it that would make it incompatible. And if you really need that complexity, it's usually worth whipping up a parser for.
"find" is a poor man's "structured-grep." It provides a bunch of functionality for filtering a result set, but is totally specific to lists of files. You can't use find with ps, netstat, iptables, ifconfig, or any other command-line program that produces a list of records. > And if you really need that complexity, it's usually worth whipping up a parser for. No work is worth doing if it could just as easily have been…
in a way, the 'find' program is like what you envision, except it's just for files. That means that someone somewhere along the road, had the same idea/problem (but limited to files) as you and whipped up a parser to produce that meta-data. That particular parser proved to be so useful to so many people, it became it's own program.
There's more than 40 years of sofware-"evolution" contained in unix, and apparently retrieving structured on the command-line has only proven universally useful for files. Unix has outlived many at the time more modern operating systems, and I think it's partly because it lacked a "grand unifying vision". Instead it has a "small, quick&dirty unifying vision" of which "flat text processing" on the commandline is a central part. It has turned out to be the greatest common denominator for being able to write programs, that might be quick and dirty oneliners, but ultimately they got the job done. And only those tiny little utilities that proved to be universally useful were developed into bigger more stuctured programs.
I'm not saying you idea is without merit, but it does apply the principle of "this concept A is useful for this particular problem-set. Let's apply it natively to all problem-sets so it can be useful there too!" ( in a way like Java did with the OO concept).
When simpler visions and concepts are actually implemented in the end it usually turns out one has been replacing witchcraft with voodoo.
Earlier quoted context omitted.
Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?
It's true that many Protocol Buffer libraries include representations, but these are for convenience; Protocol Buffers are defined in terms of their serialization format and schema.
To be super concrete, I can't read a file containing protos without knowing their type, what fields they contain, etc. I can however read JSON just fine without knowing the precise schema being encoded.
Earlier quoted context omitted.
I played with SNOBOL. It may have a more powerful matching engine, but power doesn't equate to usability. Having each line followed by 3 gotos does not good UX make.
Icon or its descendant Unicon might be worth a look, then.
It's the same story with Pike's paper. He's touting his own creation: the Sam editor.
People have strong biases toward what software they want to use. Regardless of whether it is actually superior or not. It's not about power (=execution speed and amount of code). It's about fitting someone's idea of what power is. That can vary.
Look at the people menitioning "Powershell" in this thread. It does not matter if it's inferior to UNIX (or just a lame copy of ksh, 20 years too late). If people like it, they'll use it. And they'll believe it is better than the alternatives.
Earlier quoted context omitted.
offtopic: sam(and acme) sounds like an interesting method at first, but when you try it out, its really weird and slow to get stuff done. Its because for me, mouse is really inferior to keyboard to do the majority of the tasks. This is an except from coders at work: Seibel: Is there anything you would have done differently about learning to program? Do you have any regrets about the sort of path you took or do you wi…
Acme mouse chording is really cool, doing something similar with keyboard-oriented editors is much more tedious and often outright painful ( http://acme.cat-v.org/mouse ). I suspect ken spends most of his time thinking rather than typing, and I have found this to be true of most great hackers. Other famous Sam users include Brian Kernighan, Bjarne Stroustrup(!) and Tom Duff. Kernighan writes at least as much English…
Also, a tiny program that does not require lots of lines can fit in a chip's cache, and stay there. Can't say the same for large programs.
Acme and Sam depend on GUI's. What happens when you can't get into the GUI? That never happens, right?
I would not expect these old guys to want to work too hard these days. They paid their dues. Mouse clicking sounds about right.
The real problem is that Unix commands produce flat text output without any information about how to parse that text back into structured data. Any user who wants the structured version of the data has to parse it themselves, but these parsers are ad hoc and incomplete by their very nature. People praise perl, sed, awk, cut, etc. for being good at text processing. But the only reason they need these text-processing t…
The real problem is that Unix commands produce flat text output without any information about how to parse that text back into structured data. Any user who wants the structured version of the data has to parse it themselves, but these parsers are ad hoc and incomplete by their very nature. There are a variety of serialization schemes that are quite easy to parse and would be suitable for the output of most Unix comm…
Earlier quoted context omitted.
I like your observation but I don't like your conclusion. I think the major thing missing from the world is easy piping a la Unix. You can't pipe your list of paying customer's email addresses to paypal at the command line with some switches - really, no matter how many switches you use - and bill each one the amount stated, because "paypal" is not a commandline app. You can't pipe the results of some long-running an…
I could not agree more with your comment. There's not a problem with text. Why do people pretend there is? They often only achieve making other people's jobs more difficult. Text is what people can read. People do not read binary. When something goes wrong, debugging binary formats becomes insanely cumbersome. The concept of lines is a human one. It is how humans parse. If humans could parse without needing the conce…
Though my boss did impress me by watching an OSI transaction in flight on our network monitor stop it and point to a dword and say thats wong and its Sprints broken x.400 implimentation.
Earlier quoted context omitted.
I like your observation but I don't like your conclusion. I think the major thing missing from the world is easy piping a la Unix. You can't pipe your list of paying customer's email addresses to paypal at the command line with some switches - really, no matter how many switches you use - and bill each one the amount stated, because "paypal" is not a commandline app. You can't pipe the results of some long-running an…
> The direction you're suggesting we take things is, in fact, a fuller API. Nothing about my proposal has anything to do with an API. I'm just proposing a structured stream of data instead of an unstructured one. > They're slower and worse. I am proposing avoiding a serialize/parse step between every pair of pipeline elements (or using a more structured/optimized format if a serialization step is desired). Doing less…
Right. So how does grep know it's talking to ls about dir contents with a (disk) size field and not talking to ps about process table contents with a (mem) size field? Some sort of slower content negotiation, I presume.
> cannot possibly
I suggest that every time you think that, you double-check your assumptions. In this case, I think you're just pushing all the extra hard work off onto some other part of the algorithm.
I ran into a wall when trying to use unix-y tools to do somewhat complex, regex/replace functions for code refactors. Basically, anything inside a single line is easy, but once you cross that line barrier, the complexity increases dramatically. That rendered the changes useless because most programming languages allow you to add arbitrary new-lines between any token in the language.
1. If it's longer than 15 lines, rewrite it in {python,ruby,perl} 2. Never accidentally rewrite grep, sed or awk.