Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use Haskell for shell scripting

haskellforall.com

Post reply on HN
Print view View original

81–90 of 105 posts

Re: Use Haskell for shell scripting

#81

Post by mercurial »

Gabriel439 wrote: post #70
mercurial wrote: post #26

Earlier quoted context omitted.

That's not the issue. The issue is that, if you want to simulate both "grep" and "grep -r", you need to different functions, or you need to have your "grep" function accept a record of parameters.

Actually, you can do `grep -r` by just combining `grep` and `lstree`. Here's an example: example = do file This is an example of how most of Bash's option heavy ecosystem is an outgrowth of Bash's limitation as a language (individual commands accumulate flags to work around functionality difficult to implement within the Host language). I think having a decent host language decreases the need for so many configuratio…

You're right to some extent, but I think many of these 'knobs' have a good reason to exist (--dry-run, -a, -z for rsync for instance) and cannot be usefully, or at all, replaced by more composability. And attempting to implement support for them will run against the limitations of Haskell's syntax.

Something like OCaml would be better suited, since polymorphic variants, named and default arguments give a lot more flexibility, though the fact that shell commands happily return different outputs depending on their options would still be an issue.

Re: Use Haskell for shell scripting

#82

Post by thinkpad20 »

Gabriel439 wrote: post #67
barrkel wrote: post #25

OK. How do you easily fork to run a command in the background? How does setting up pipes work? What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done (I'd use a subshell with (ch xxx; ...) in bash)? Getting into more tricky stuff, what's the equivalent of This doesn't really demonstrate anything that shell scripts are actually written for: orchestrating and composing other…

> How do you easily fork to run a command in the background? `turtle` provides `fork` for running a command in the background. Example usage: example = do using (fork commandToForkInAnotherThread) theseCommandsStillRunInTheOriginalThread > How does setting up pipes work? See the `inproc` and `inshell` commands, which let you convert any shell command into a stream transformation embedded within Haskell. > What's the…

In my opinion it be nice to write it as a wrapper type thing:

    inDir d action = do
      oldDir
Then you could use it like a python `with` statement. :)

Re: Use Haskell for shell scripting

#83

Post by mercurial »

vlastachu wrote: post #46
mercurial wrote: post #26

Earlier quoted context omitted.

That's not the issue. The issue is that, if you want to simulate both "grep" and "grep -r", you need to different functions, or you need to have your "grep" function accept a record of parameters.

I'm not good enough haskell programmer, but there is possible solution (records as you mentioned). import Prelude hiding ((-)) data Grep = Grep {isRecursive :: Bool, maxCount :: Maybe Int} --etc deriving (Show) grep = Grep False Nothing --short pseudonim r :: Grep -> Grep r command = command{isRecursive = True} m :: Int -> Grep -> Grep m num command = command{maxCount = Just num} (-) :: a -> (a -> a) -> a (-) command…

Yes, that's exactly what I wouldn't want to type. Also, your record is going to blow up in the likely case another utility uses a recursive flag, because of Haskell's pervasive namespacing problems.

Re: Use Haskell for shell scripting

#84

Post by Gabriel439 »

cies wrote: post #75
Gabriel439 wrote: post #65

Earlier quoted context omitted.

You use `inproc` and `inshell` for piping. For example, here's the type of `inshell`: inshell :: Text -- Shell command -> Shell Text -- Standard input to feed command -> Shell Text -- Standard output produced by command I made one intentional simplification in the API, which was to not provide a way to capture standard error. It's definitely possible to provide such a utility, but I wanted to simplify things as much…

Thanks for this elaborate response.

You're welcome!

Re: Use Haskell for shell scripting

#85

Post by tel »

mercurial wrote: post #83
vlastachu wrote: post #46

Earlier quoted context omitted.

I'm not good enough haskell programmer, but there is possible solution (records as you mentioned). import Prelude hiding ((-)) data Grep = Grep {isRecursive :: Bool, maxCount :: Maybe Int} --etc deriving (Show) grep = Grep False Nothing --short pseudonim r :: Grep -> Grep r command = command{isRecursive = True} m :: Int -> Grep -> Grep m num command = command{maxCount = Just num} (-) :: a -> (a -> a) -> a (-) command…

Yes, that's exactly what I wouldn't want to type. Also, your record is going to blow up in the likely case another utility uses a recursive flag, because of Haskell's pervasive namespacing problems.

It probably wouldn't be too hard to do something like

    grep "foo"
      & "recursive"
in a typesafe way. It just probably wouldn't be worth the complexity. It also probably couldn't be a straight `IO` action then, which was a design constraint of Gabriel's.

Re: Use Haskell for shell scripting

#86

Post by kenko »

Gabriel439 wrote: post #65
cies wrote: post #9

Thanks Gabriel Gonzalez! There is a comment on the blog post (by Chris Done) asking how it deals with piping. I really wonder about that too. Some related projects: - Joey Hess recently released a nice Haskell-to-sh compiler. I like this approach as the resulting sh scripts are runnable on pretty much every *nix. https://joeyh.name/blog/entry/shell_monad/ - Chris Done also released a lib to do shell stuff from Haskel…

You use `inproc` and `inshell` for piping. For example, here's the type of `inshell`: inshell :: Text -- Shell command -> Shell Text -- Standard input to feed command -> Shell Text -- Standard output produced by command I made one intentional simplification in the API, which was to not provide a way to capture standard error. It's definitely possible to provide such a utility, but I wanted to simplify things as much…

Why `Either Text Text`? What if you're interested in both stdout and stderr?

Re: Use Haskell for shell scripting

#87

Post by mightybyte »

mercurial wrote: post #21

I like the Pattern thing. However, it seems to me that you're going to quickly run into trouble if you need to even vaguely emulate shell scripting. Shell utilities live and die by their options. It's unfortunate Haskell supports neither named arguments nor default values. Which means that in order to emulate options, you would need to pass records to your "shell" utility, which, on top of being cumbersome, forces yo…

It's going to be difficult for something like this to match the ease of shell scripting. For instance, typing `cd "foo"` is significantly more painful than `cd foo`. (Maybe this could be fixed by forking or adding to ghci so that when you hit space after the function name it automatically puts the quotes in for you and places the cursor between them.) Options are certainly important, and as others have said, they can be emulated with records. Is the syntax going to just as convenient as shell scripting? No. But that's not the point. The point is that shell scripting is massively painful in a lot of other ways where Haskell blows it away. So the task is to find a happy medium that gets fairly close to the convenience of shell scripting while still giving us the power of Haskell.

There are a number of potential approaches for coming close to the ease of shell scripting. One is options records as others have mentioned. For defaults you can have a Default instance (no, that's not boilerplate because you would have had to specify the defaults somewhere anyway). Then there is plenty of room for infix operator combinators to make it easier to change individual options. A second option could be to put options into a string that would get parsed into a record. You could use patterns similar to those used in existing command line argument processors like optparse-applicative. Or, if you don't like that, then maybe a quasiquote could give more power.

Do these things require some boilerplate? Yes. We know that is going to be required since Haskell wasn't designed for the convenience that shells were designed for. But that's fine in this case because the potential benefits are huge.

Re: Use Haskell for shell scripting

#88

Post by npsimons »

Nice! Now I can add Haskell to my list of languages I can script with.

I'm always on the lookout for new languages I can script with (or at least get closer to rapid prototyping) for easier learning, testing, problem solving, etc. I've got templates that I run against linters, style checkers, etc for many languages and it will be helpful to have even more options.

Re: Use Haskell for shell scripting

#89

Post by npsimons »

dkarapetyan wrote: post #4

Who's the target audience of this exactly? I already see a language pragma, do notation, liftIO, parser combinators. Hamming has this great set of lectures on how he became a world renowned scientist and in one of the lectures he explains why Ada failed and other languages succeeded. The difference was that Ada was designed logically and most successful languages were designed psychologically. Even when government co…

As a working programmer and a bit of a language geek, a few years ago I decided to try to get as many programming languages as possible installed on my dev machines. From this, I eventually tried to get as many of them "scriptable" as possible, creating at a bare minimum a "Hello, world!" template that could be run from the command line. I like having options, or a bare minimum, having things around to play with/learn when I've got down time.

I remember something in "Mythical Man-Month" that extolled the virtues of scripting programming for concept exploration, and I've often felt this is one of the major advantages traditionally scriptable languages have over compiled. Once you can run a program without compiling it, iteration tends to go faster.

So sure, some languages require more boilerplate to get started than others, but I've got templates for that, and I happily scripted almost all the exercises in "Thinking in C++" because it just made working them out faster, even in emacs where I can bind the compile key to any command I can dream of.

Re: Use Haskell for shell scripting

#90

Post by Gabriel439 »

barrkel wrote: post #72
tinco wrote: post #40

Earlier quoted context omitted.

Did we read the same article? The entire 'streaming section' is about pipes and I/O redirects. Running a command in the background is just forkIO $ proc ..etc.., as in regular Haskell. Nothing tricky about it.

The streaming section of the article has nothing about composing processes, that I could see; it appeared to be about treating the output of commands as input to Haskell lazy lists. I may have misread it, though. Here's a pattern that comes up fairly frequently for me: foo | fgrep -v -f It uses the second column in info.csv as fixed strings to match inside lines in the output of foo, and filters them out, with the re…

There are two ways you can embed that within `turtle`. You can either embed each step as its own concurrent process, like this:

    -- Note, the flow is right-to-left, not left-to-right
    inshell "bar" (inshell "fgrep ..." (inshell "foo" empty))
Or you can just embed the entire thing within a single `inshell` command:

    inshell "foo | fgrep ... | bar"
The reason this works is that the type of `inshell` is:

    inshell
        :: Text        -- Command line
        -> Shell Text  -- Standard input to feed
        -> Shell Text  -- Standard output from command
This leads you stream to any shell command's input and read the command's output also as a stream.
Post reply on HN