Live data from Hacker News

For the Love of Pipes

blog.jessfraz.com

301–310 of 323 posts

Re: For the Love of Pipes

#301
post #292
post #83

Earlier quoted context omitted.

> The Unix philosophy is documented by Doug McIlroy as TaoUP has a longer discussion[1] of the Unix philosophy, which includes Rob Pike's and Ken Thompson's comments on the philosophy. [1] http://www.catb.org/esr/writings/taoup/html/ch01s06.html "Those who don't understand Unix are condemned to reinvent it, poorly." (Henry Spencer)

Avoid TAOUP, is really bad. Most of the lore have stolen from places as LISP and VAX communities. ESR is to Pike and Ken as alien as X11 itself.

I enjoyed it when I read it many years ago, but maybe that was because I was inexperienced and naive.

Could you recommend some "original sources" to learn from, instead? Ideally in book form?

Re: For the Love of Pipes

#302
post #293

Earlier quoted context omitted.

In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place. Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change write'. So I like to write pipelines like this: cat foo…

awk can do all of that except sed. And I am not sure about the last. No need to wc ($NF in AWK, if I can recall), no need for grep, you have the /match/ statement, with regex too.

> except sed

Doesn't gsub(/a/, "b") do the same thing as s/a/b/g?

Re: For the Love of Pipes

#303
post #292
post #83

Earlier quoted context omitted.

> The Unix philosophy is documented by Doug McIlroy as TaoUP has a longer discussion[1] of the Unix philosophy, which includes Rob Pike's and Ken Thompson's comments on the philosophy. [1] http://www.catb.org/esr/writings/taoup/html/ch01s06.html "Those who don't understand Unix are condemned to reinvent it, poorly." (Henry Spencer)

Avoid TAOUP, is really bad. Most of the lore have stolen from places as LISP and VAX communities. ESR is to Pike and Ken as alien as X11 itself.

I'm not sure I understand - is TAOUP bad because it stole information, or because the information it has is wrong?

Because I'd consider the latter to be far worse than the former.

Re: For the Love of Pipes

#304
post #263
post #211

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…

Don’t you have to rewrite every single program to make it able to read those “objects”?

You don't have to rewrite them; just write them. Every new OS/language needs its tooling/libraries to be written.

Re: For the Love of Pipes

#305
post #265
post #211

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…

What happens when there are upstream changes to the objects? Does everything downstream just need to change, and hence, the upstream objects returned by programs need to be changed with care? Or is it using something like protobuf, where fields are only additive, but never deleted, for backwards compatibility? Or are the resulting chain of pipes so short lived, it doesn't matter?

Powershell is a shell language, everything is dynamic, and does not error when you access a non-existent property on an object.

Re: For the Love of Pipes

#306

Earlier quoted context omitted.

> list.select { |x| x.foo > 10 }.map { |x| x.bar }... Forgive me if I'm misreading this syntax, but to me this looks like plain old function composition: a call to `select` (I assume that's like Haskell's `filter`?) composed with a call to `map`. No monad in sight. As I mentioned, monads are more about collapsing structure. In the case of lists this could be done with `concat` (which is the list implementation of mon…

Nope, it's not. It's Ruby, and the list could be an eager iterator, an actual list, a lazy iterator, a Mabye (though it would be clumsy in Ruby), etc. And monads are not "more about collapsing structure". They are just a design pattern that follows a handful of laws. It seems like you're mistaking their usefulness in Haskell for what they are. A lot of other languages have monads either baked in or an element of the…

> It's Ruby

Thanks for clarifying; I've read a bunch of Ruby but never written it before ;)

From a quick Google I see that "select" and "map" do work as I thought:

https://ruby-doc.org/core-2.2.0/Array.html#method-i-select

https://ruby-doc.org/core-2.2.0/Array.html#method-i-map

So we have a value called "list", we're calling its "select" method/function and then calling the "map" method/function of that result. That's just function composition; no monads in sight!

To clarify, we can rewrite your example in the following way:

    list.select { |x|  x.foo > 10 }.map { |x| x.bar }

    # Define the anonymous functions/blocks elsewhere, for clarity
    list.select(checkFoo).map(getBar)

    # Turn methods into standalone functions
    map(select(list, checkFoo), getBar)

    # Swap argument positions
    map(getBar, select(checkFoo, list))

    # Curry "map" and "select"
    map(getBar)(select(checkFoo)(list))

    # Pull out definitions, for clarity
    mapper   = map(getBar)
    selector = select(checkFoo)
    mapper(selector(list))
This is function composition, which we could write:

    go = compose(mapper, selector)
    go(list)
The above argument is based solely on the structure of the code: it's function composition, regardless of whether we're using "map" and "select", or "plus" and "multiply", or any other functions.

To understand why "map" and "select" don't need monads, see below.

> the list could be an eager iterator, an actual list, a lazy iterator, a Maybe (though it would be clumsy in Ruby), etc.

Yes, that's because all of those things are functors (so we can "map" them) and collections (so we can "select" AKA filter them).

The interface for monad requires a "wrap" method (AKA "return"), which takes a single value and 'wraps it up' (e.g. for lists we return a single-element list). It also requires either a "bind" method ("concatMap" for lists) or, my preference, a "join" method ("concat" for lists).

I can show that your example doesn't involve any monads by defining another type which is not a monad, yet will still work with your example.

I'll call this type a "TaggedList", and it's a pair containing a single value of one type and a list of values of another type. We can implement "map" and "select" by applying them to the list; the single value just gets passed along unchanged. This obeys the functor laws (I encourage you to check this!), and whilst I don't know of any "select laws" I think we can say it behaves in a reasonable way.

In Haskell we'd write something like this (although Haskell uses different names, like "fmap" and "filter"):

    data TaggedList t1 t2 = T t1 [t2]

    instance Functor (TaggedList t1) where
      map f (T x ys) = T x (map f ys)

    instance Collection (TaggedList t1) where
      select f (T x ys) = T x (select f ys)
In Ruby we'd write something like:

    class TaggedList
      def initialize(x, ys)
        @x  = x
        @ys = ys
      end

      def map(f)
        TaggedList.new(@x, @ys.map(f))
      end

      def select(f)
        TaggedList.new(@x, @ys.select(f))
      end
    end
This type will work for your example, e.g. (in pseudo-Ruby, since I'm not so familiar with it):

    myTaggedList = TaggedList.new("hello", [{foo: 1, bar: true}, {foo: 20, bar: false}])
    result = myTaggedList.select { |x|  x.foo > 10 }.map { |x| x.bar }

    # This check will return true
    result == TaggedList.new("hello", [false])
Yet "TaggedList" cannot be a monad! The reason is simple: there's no way for the "wrap" function (AKA "return") to know which value to pick for "@x"!

We could write a function which took two arguments, used one for "@x" and wrapped the other in a list for "@ys", but that's not what the monad interface requires.

Since Ruby's dynamically typed (AKA "unityped") we could write a function which picked a default value for "@x", like "nil"; yet that would break the monad laws. Specifically:

    bind(m, wrap) == m
If "wrap" used a default value like "nil", then "bind(m, wrap)" would replace the "@x" value in "m" with "nil", and this would break the equation in almost all cases (i.e. except when "m" already contained "nil").

Re: For the Love of Pipes

#307
post #253
post #154

Earlier quoted context omitted.

I find something like this: grep '^x' to be very readable, as the flow is still visually apparent based on punctuation.

In this particular example, ‘unnecessary use of cat’ is accompanied by ‘unnecessary use of grep’. cat input | grep '^x' | sed 's/foo/bar/g' → sed '/^x/s/foo/bar/g'

That's not the same thing. The sed output will still keep lines not starting with x (just not replacing foo with bar in those) where grep will filter those out.

Re: For the Love of Pipes

#308
post #280
post #211

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…

Never parse ls.

> Never parse ls.

I have heard this several times, but either I do not understand it or I disagree. Do you mean parsing the output of the ls program? Parsing ls output is not wrong, the program produces a text stream that is easy and useful to parse. There's nothing to be ashamed when doing it, even when you can do it in a different, even shorter way. I do grep over ls output daily, and I find it much more convenient than writing wildcards.

Re: For the Love of Pipes

#309
post #253

Earlier quoted context omitted.

In this particular example, ‘unnecessary use of cat’ is accompanied by ‘unnecessary use of grep’. cat input | grep '^x' | sed 's/foo/bar/g' → sed '/^x/s/foo/bar/g'

That's not the same thing. The sed output will still keep lines not starting with x (just not replacing foo with bar in those) where grep will filter those out.

Yeah, Muphry's law at work. Corrected version:

   sed -n '/^x/s/foo/bar/gp' 
This may be an inadvertent argument for the ‘connect simpler tools’ philosophy.

Re: For the Love of Pipes

#310
post #261

Earlier quoted context omitted.

> I would hate to see the day HN allowed any way to bold sections of text. HN already has shitty italics (shitty in that it commonly matches and eats things you don't want to be italicised e.g. multiplications, pointers, … in part though not only because HN doesn't have inline code). "bold" can just be styled as italics, or it can be styled as a medium or semibold. It's not an issue, and even less worth it given how…

For a site that's meant to target programmers, HN's handling of code blocks is pretty poor. Just give me the triple-tilde code block syntax please!

> For a site that's meant to target programmers, HN's handling of code blocks is pretty poor.

Meh. It does literal code blocks, they work fine.

That's pretty much the only markup feature which does, which is impressively bad given HN only has two markup feature: literal code blocks and emphasis.

It's not like they're going to add code coloration or anything.

And while fenced code blocks are slightly more convenient (no need to indent), pasting a snippet in a text editor and indenting it is hardly a difficult task.

Post reply on HN