Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use Haskell for shell scripting

haskellforall.com

Post reply on HN
Print view View original

61–70 of 105 posts

Re: Use Haskell for shell scripting

#61

Post by joelwilliamson »

vlastachu wrote: post #51
mercurial wrote: post #50

Earlier quoted context omitted.

> If your arguments aren't exclusive, you can pass a list of algebraic data. Which you need to prefix to avoid clashes. > If they are exclusive, you can construct a type for them. Which is going to end up being a record, which: - is awkward to build (compared to just giving options to a command or arguments to a function) - will most likely need to be an instance of Default - which needs to have its fields prefixed t…

> prefixes, default instances Also bad setters ("record {field = val}" looks nice but useless, as not a function). I hope the developers of GHC also clearly see the problem and someday will be engaged in it. Then the language becomes much more expressive. Is that task exists somewhere in roadmap?

There has been quite a bit of discussion on ghc-devs about fixing records. The most promising approach seems to here:

http://nikita-volkov.github.io/record/

Re: Use Haskell for shell scripting

#62

Post by mightybyte »

S4M wrote: post #14

I don't know much about Haskell, but I thought it had some properties to isolate side effects, but the code he gives: main = do cd "/tmp" mkdir "test" output "test/foo" "Hello, world!" -- Write "Hello, world!" to "test/foo" stdout (input "test/foo") -- Stream "test/foo" to stdout rm "test/foo" rmdir "test" sleep 1 die "Urk!" Clearly doesn't (it creates a directory, writes in a file, removes that file and that directo…

That's because the whole block you're pointing to there is in Haskell's side effects box. So this is simply not the right example for illustrating how Haskell does isolate side effects. Shell scripts in general are very side effecting, so for this application it makes sense.

Re: Use Haskell for shell scripting

#63

Post by danidiaz »

barrkel wrote: post #25

OK. How do you easily fork to run a command in the background? How does setting up pipes work? What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done (I'd use a subshell with (ch xxx; ...) in bash)? Getting into more tricky stuff, what's the equivalent of This doesn't really demonstrate anything that shell scripts are actually written for: orchestrating and composing other…

> What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done

This can be done with the "bracket" function, which works roughly like a context manager in Python:

  import Control.Exception
  import System.Directory

  withDirectory :: FilePath -> IO a -> IO a
  withDirectory path action = bracket (getCurrentDirectory

Re: Use Haskell for shell scripting

#64

Post by klibertp »

loudmax wrote: post #58
chrisBob wrote: post #52

After learning Perl I started using it where some more educated people might recommend a proper shell script. My thinking is that using what you know is a whole lot more efficient than learning a new tool for a small job, even if some people think it is the right tool. I am sure it is no different for people familiar with Haskell.

I do a lot of shell scripting, and I'm not sure there is such a thing as a "proper" shell script. The shell just isn't a great programming language. Just about any modern scripting language is better, starting with Perl. But the shell has been the lingua franca of the Unix world for decades now. It's the one language that you can pretty much guarantee is on any Unix or Linux server, even pretty ancient ones. I don't…

> you can pretty much guarantee is on any Unix or Linux server, even pretty ancient ones

Well, yes and no. You can get reasonable compatibility with different Unix flavours if you stick to sh. Your script is not going to work on BSDs once you start using bash specific features, though.

Fun fact: on FreeBSD bash does not live in /bin/bash, it's in /usr/local/bin/bash. Every time you write a shebang with /bin/bash hardcoded you're making your script harder to use there.

Perl is everywhere almost by default and it's more compatible as it has just one implementation, without sh/bash/csh/ksh/tcsh/zsh madness. I'd say it's a good idea to use Perl instead of shell script for anything more complicated than a few lines of code if it's meant to be portable. (And I'm not Perl programmer at all).

Re: Use Haskell for shell scripting

#65

Post by Gabriel439 »

cies wrote: post #9

Thanks Gabriel Gonzalez! There is a comment on the blog post (by Chris Done) asking how it deals with piping. I really wonder about that too. Some related projects: - Joey Hess recently released a nice Haskell-to-sh compiler. I like this approach as the resulting sh scripts are runnable on pretty much every *nix. https://joeyh.name/blog/entry/shell_monad/ - Chris Done also released a lib to do shell stuff from Haskel…

You use `inproc` and `inshell` for piping. For example, here's the type of `inshell`:

    inshell
        :: Text        -- Shell command
        -> Shell Text  -- Standard input to feed command
        -> Shell Text  -- Standard output produced by command
I made one intentional simplification in the API, which was to not provide a way to capture standard error. It's definitely possible to provide such a utility, but I wanted to simplify things as much as possible in the first release before the slow onslaught of feature cruft begins. If there were such a utility, it would have this type:

    both
        :: Text        -- Shell command
        -> Shell Text  -- Standard input to feed command
        -> Shell (Either Text Text)
... and you could selectively listen to just stderr or stdout by taking advantage of the fact that pattern match failures short-circuit downstream commands:

    Left txt
There is one more shell library that I know of: `process-streaming`. I actually didn't know about `shell_monad` (that's the one most similar in spirit to what I wrote).

The main reason I rolled my own library is that this was written with the specific audience of people who didn't know any Haskell, but were comfortable with Python or Bash. My actual goal is to convince people internally at Twitter to use Haskell instead of Python for large scripts. I reviewed all those libraries (with the exception of shell_monad) to see if I felt comfortable marketing them to non-Haskell programmers and none of them felt like the right level of abstraction to me. I almost ended up going with Shelly, but in the process of polishing shelly for internal usage I found myself continually wrapping things with better names, different types, and providing missing features to get a single import umbrella, so I just stopped and asked: "why not just do this as a cohesive single library instead?". Also, `shelly` does not provide any `IO`-only commands: everything has to be wrapped in the `Sh` monad.

As for the other libraries, `shell-conduit` was too complex for new users in my opinion and `hell` is not embedded within Haskell (it's a separate language), and I wanted to keep the features of Haskell. I still need some more time to review `shell_monad` to see if I made a mistake by ignoring it.

Re: Use Haskell for shell scripting

#66

Post by Gabriel439 »

loudmax wrote: post #58
chrisBob wrote: post #52

After learning Perl I started using it where some more educated people might recommend a proper shell script. My thinking is that using what you know is a whole lot more efficient than learning a new tool for a small job, even if some people think it is the right tool. I am sure it is no different for people familiar with Haskell.

I do a lot of shell scripting, and I'm not sure there is such a thing as a "proper" shell script. The shell just isn't a great programming language. Just about any modern scripting language is better, starting with Perl. But the shell has been the lingua franca of the Unix world for decades now. It's the one language that you can pretty much guarantee is on any Unix or Linux server, even pretty ancient ones. I don't…

Note that you only need `/usr/bin/env runhaskell` if you want to interpret the script. You can also compile the script as a native binary, which is the recommended approach on Windows.

Re: Use Haskell for shell scripting

#67

Post by Gabriel439 »

barrkel wrote: post #25

OK. How do you easily fork to run a command in the background? How does setting up pipes work? What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done (I'd use a subshell with (ch xxx; ...) in bash)? Getting into more tricky stuff, what's the equivalent of This doesn't really demonstrate anything that shell scripts are actually written for: orchestrating and composing other…

> How do you easily fork to run a command in the background?

`turtle` provides `fork` for running a command in the background. Example usage:

    example = do
        using (fork commandToForkInAnotherThread)
        theseCommandsStillRunInTheOriginalThread
> How does setting up pipes work?

See the `inproc` and `inshell` commands, which let you convert any shell command into a stream transformation embedded within Haskell.

> What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done (I'd use a subshell with (ch xxx; ...) in bash)?

You can write a combinator for this using `turtle` pretty easily:

    pushd newDir = do
        oldDir
... and you use it like this:

    example = do
        popDir
> what's the equivalent of `inproc`/`inshell` which let you read in a command's standard output as a stream

Re: Use Haskell for shell scripting

#68

Post by Gabriel439 »

dkarapetyan wrote: post #4

Who's the target audience of this exactly? I already see a language pragma, do notation, liftIO, parser combinators. Hamming has this great set of lectures on how he became a world renowned scientist and in one of the lectures he explains why Ada failed and other languages succeeded. The difference was that Ada was designed logically and most successful languages were designed psychologically. Even when government co…

The target audience is non-Haskell programmers, and if you don't think the tutorial is good enough to onboard such a programmer then I consider that a bug against the library. I would actually appreciate if people submitted Github issues highlighting any pedagogical problem with the tutorial.

I think the use of `liftIO` is a reasonable objection. When I wrote the library I had the choice of utomatically pre-wrapping all `IO` commands with `liftIO` for the user (making them all `Shell` commands) by default. However, I decided not to do that for two reasons:

* If you do that you can't use them outside of a `Shell` any longer * The user has to learn `liftIO` anyway if they want to use `IO` actions not provided by the `turtle` library. I didn't want to teach the user a leaky abstraction

I don't see any issue with `do` notation is bad. Same thing with parser combinators, which are just strings in the simple case, and the "Patterns" section of tutorial has a table showing you how to convert regular expression idioms to `Pattern`s:

http://hackage.haskell.org/package/turtle-1.0.0/docs/Turtle-...

The language pragma is sort of a grey area. I decided to keep it because it doesn't take a long time to explain and it significantly increases the usability of the library.

Re: Use Haskell for shell scripting

#69

Post by [deleted] »

Gabriel439 wrote: post #67
barrkel wrote: post #25

OK. How do you easily fork to run a command in the background? How does setting up pipes work? What's the idiom for chdir'ing to a subdirectory such that you pop back out again when you're done (I'd use a subshell with (ch xxx; ...) in bash)? Getting into more tricky stuff, what's the equivalent of This doesn't really demonstrate anything that shell scripts are actually written for: orchestrating and composing other…

> How do you easily fork to run a command in the background? `turtle` provides `fork` for running a command in the background. Example usage: example = do using (fork commandToForkInAnotherThread) theseCommandsStillRunInTheOriginalThread > How does setting up pipes work? See the `inproc` and `inshell` commands, which let you convert any shell command into a stream transformation embedded within Haskell. > What's the…

[deleted]

Re: Use Haskell for shell scripting

#70

Post by Gabriel439 »

mercurial wrote: post #26
gamegoblin wrote: post #23

Earlier quoted context omitted.

Quite a lot of libraries here: https://wiki.haskell.org/Command_line_option_parsers

That's not the issue. The issue is that, if you want to simulate both "grep" and "grep -r", you need to different functions, or you need to have your "grep" function accept a record of parameters.

Actually, you can do `grep -r` by just combining `grep` and `lstree`. Here's an example:

    example = do
        file
This is an example of how most of Bash's option heavy ecosystem is an outgrowth of Bash's limitation as a language (individual commands accumulate flags to work around functionality difficult to implement within the Host language). I think having a decent host language decreases the need for so many configuration knobs for every command.
Post reply on HN