Live data from Hacker News

Use Haskell for shell scripting

haskellforall.com

91–100 of 105 posts

Re: Use Haskell for shell scripting

#91
For me combing the best parts of bash and ipython is the way to go. Up to now this seems more comfortable to me than using subprocess in python or this haskell aproach which needs to be aware of every programme output to give what it promises. You can easily copy big parts of existing bash scripts and e.g. add error handling in the python way :) Even I think for loops/list comprehensions are betten than the strange bash syntax.

And here a short example::

  #!/usr/bin/env ipython3
  #
  # 1. echo "#!/usr/bin/env ipython3" > scriptname.ipy    # creates new ipy-file
  #
  # 2. chmod +x scriptname.ipy                            # make it executable
  #
  # 3. starting with line 2, write normal python or do some of
  #    the ! magic of ipython, so that you can use shell commands
  #    within python and even assign their output to a variable via
  #    var = !cmd1 | cmd2 | cmd3                          # enjoy ;)
  #
  # 4. run via ./scriptname.ipy - if it fails with recognizing % and !
  #    but parses raw python fine, please check again for the .ipy suffix which must be there!
  #
  # ugly example, please go and find more in the wild
  files = !ls *.* | grep "y"
  for file in files:
    !echo $file | grep "p"
  # sorry for this nonsense example ;)
  # it's even possible to access the output of a command by outputvariable.s, .p or .n
  # see file:///usr/share/doc/ipython-doc/html/interactive/reference.html#system-shell-access
Better take a look here, it's more complete: https://blog.safaribooksonline.com/2014/02/12/using-shell-co...

Re: Use Haskell for shell scripting

#92

Please forgive my lack of familiarity with the concurrent workings of Haskell, but since the Shell streams are based off []/IO, and not Concurrent.Chan, does this mean one turtle function has to complete (and write its results to memory) before the next turtle function can run? To me, magic bits of shell scripts which turtle would need to improve upon were it to replace said scripts are not the loop constructs, condi…

The streaming section shows some examples of combining turtle functions, this will be the same as shell pipes.

There's also nothing stopping you from using forkIO to spark off a separate thread, and doing IO in multiple threads concurrently.

Haskell's IO manager allows multiple threads doing concurrent IO in what looks like an imperative, one instruction after the other manner. Instead of async callbacks like you might expect from other languages.

Re: Use Haskell for shell scripting

#93
post #76

Earlier quoted context omitted.

> How do you easily fork to run a command in the background? `turtle` provides `fork` for running a command in the background. Example usage: example = do using (fork commandToForkInAnotherThread) theseCommandsStillRunInTheOriginalThread > How does setting up pipes work? See the `inproc` and `inshell` commands, which let you convert any shell command into a stream transformation embedded within Haskell. > What's the…

FWIW: >(foo) does the same thing, except the other way around, for process output.

There is a "createPipe" function in the "unix" package http://hackage.haskell.org/package/unix-2.7.1.0/docs/System-... that gets us half-way towards process substitution.

Unfortunately, I don't know how to get the name of the device file associated to the pipe, and I need it in order to pass it as an argument to the reading process :(

Re: Use Haskell for shell scripting

#94
post #74

Earlier quoted context omitted.

> you can pretty much guarantee is on any Unix or Linux server, even pretty ancient ones Well, yes and no. You can get reasonable compatibility with different Unix flavours if you stick to sh. Your script is not going to work on BSDs once you start using bash specific features, though. Fun fact: on FreeBSD bash does not live in /bin/bash, it's in /usr/local/bin/bash. Every time you write a shebang with /bin/bash hard…

Oh that's interesting. I'd assumed that FreeBSD had made bash the default shell around the same time that Mac OS did. I guess my point still stands for /bin/sh. Not a fun programming language though.

No way. One of FreeBSD's goals is to get rid of everything that's GPL-licensed. bash is not only that, but it's also horrible code.

And it's a user-friendly shell with all the tab completions and history searches, which DOES NOT BELONG IN /bin/sh!

Re: Use Haskell for shell scripting

#95
post #74

Earlier quoted context omitted.

Oh that's interesting. I'd assumed that FreeBSD had made bash the default shell around the same time that Mac OS did. I guess my point still stands for /bin/sh. Not a fun programming language though.

No way. One of FreeBSD's goals is to get rid of everything that's GPL-licensed. bash is not only that, but it's also horrible code. And it's a user-friendly shell with all the tab completions and history searches, which DOES NOT BELONG IN /bin/sh!

Exactly, which is the reason why BSDs were not affected by shellshock. And moreover, modern tcsh is quite a powerful and full-featured shell, too.

I unfortunately had to switch to Linux a few years ago (after using FreeBSD for almost a decade) and I still miss how consistent and well laid out BSDs seem in comparison.

Re: Use Haskell for shell scripting

#96
post #86

Earlier quoted context omitted.

You use `inproc` and `inshell` for piping. For example, here's the type of `inshell`: inshell :: Text -- Shell command -> Shell Text -- Standard input to feed command -> Shell Text -- Standard output produced by command I made one intentional simplification in the API, which was to not provide a way to capture standard error. It's definitely possible to provide such a utility, but I wanted to simplify things as much…

Why `Either Text Text`? What if you're interested in both stdout and stderr?

Then you can do this:

    fmap (either id id) (both ...)
... which is equivalent to:

    x  txt
        Right txt -> txt)
That removes the `Either` tag and fuses them into a single stream.

Re: Use Haskell for shell scripting

#97
post #55
post #33

Earlier quoted context omitted.

You're right that Haskell doesn't distinguish different IO actions other than by the type they return. There are certainly libraries that do this though, although they aren't widely used. Even though Haskell doesn't distinguish different classes of IO actions, it still distinguishes IO actions from other kinds of actions (such as stateful actions as per your example), and pure computations and that provides a hell of…

> Not widely used... I use them in every single application I write, or I make my own tighter, more specific ones. They're incredibly useful in real world apps.

Sorry Tel, I didn't mean to imply that nobody uses them, more that I would guess that they are used less than 1% of the time where their inclusion could be beneficial.

Re: Use Haskell for shell scripting

#99
post #97
post #55

Earlier quoted context omitted.

> Not widely used... I use them in every single application I write, or I make my own tighter, more specific ones. They're incredibly useful in real world apps.

Sorry Tel, I didn't mean to imply that nobody uses them, more that I would guess that they are used less than 1% of the time where their inclusion could be beneficial.

Ha, no offense. I just wanted to emphasize that they are used.

In particular, I think they're more useful in applications than libraries and most Haskell code you can find in the wild is library code---so you end up not seeing them much.

Re: Use Haskell for shell scripting

#100
post #29
post #18

Earlier quoted context omitted.

You are right. In this case effects are not isolated. But in this particular script, there are no interesting things to move into a pure function. It does not mean that it wouldn't be the case in a more complex script. Like everything, you have to learn to balance your IO code and your pure code. A bit like learning when to factor something into a separate class, or leave it in a few statement/methods. If you write e…

I agree about the benefits of isolating side effects and IO - I generally code in python, and my code tends to look like: def main_function(args): data = get_data(args) result = do_calculations(data) push_results(result, args) Where the function do_calculations is somewhat pure - no side effects, but I do use local variables that I modify inside the function. > You are right. In this case effects are not isolated. Bu…

Note that style of Pyhton uses block/strict IO (not streaming/lazy) which makes it rather inefficient if any part of he process exits early (due to error or because the user only wanted the first line of output). Most common command line programs (and simple Haskell programs) are streaming not block.
Post reply on HN