Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ctypes.sh – A foreign function interface for bash

ctypes.sh

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Ctypes.sh – A foreign function interface for bash

#21

Post by AndyKelley »

adrusi wrote: post #12
AndyKelley wrote: post #11

The problem with this project is that somebody might use it. I argue that if you need to do something in bash that involves more than a sequence of commands, including a simple if statement, then you should switch from bash to a proper scripting language (python, perl, ruby, nodejs, take your pick.)

I disagree. Shell scripts are great for automating system tasks — for sysadminy type stuff, including tasks with high complexity with nested loops and conditionals. A scripting language like those you mentioned is great for if you need to run your script on systems you don't control, or you need it to be cross platform, or you'd describe what you're doing as creating software rather than automating tasks. No scriptin…

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

Re: Ctypes.sh – A foreign function interface for bash

#22

Post by AndyKelley »

adrusi wrote: post #20
dragonwriter wrote: post #13

Earlier quoted context omitted.

> No scripting language can ever integrate with the host system quite as well as shell scripts can. What is the basis for this? Shell script is a scripting language (more precisely, a set of scripting languages with similar features.) The difference between shell scripting languages and other scripting languages is that the former are optimized around the need to scale down to a convenient line-by-line way to work wi…

In addition to AnimalMuppet's point, shell scripts make no attempt to be cross platform, so you don't get any awkward leaky abstractions when trying to interact with low-level system facilities. The primitives in shell scripts are the primitives of the operating system. The fundamental building blocks of your language are strings and files and processes, which makes it really convenient to work with strings and files…

You can easily invoke platform-specific binaries from within a scripting language. You can even invoke them in a shell if you want. Check out this API: https://docs.python.org/3/library/subprocess.html#subprocess...

Re: Ctypes.sh – A foreign function interface for bash

#23

Post by e12e »

AndyKelley wrote: post #21
adrusi wrote: post #12

Earlier quoted context omitted.

I disagree. Shell scripts are great for automating system tasks — for sysadminy type stuff, including tasks with high complexity with nested loops and conditionals. A scripting language like those you mentioned is great for if you need to run your script on systems you don't control, or you need it to be cross platform, or you'd describe what you're doing as creating software rather than automating tasks. No scriptin…

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

> Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

Install perl/python/ruby and nodejs -- requiring just the shell to be installed?

Snark aside (it's not only snark - pretty much every system will have something like a posix shell), proper posix portable shell is hard - it's an old gnarly language -- but it's what we've got. And with a bit of discipline and good practices -- it doesn't have to be bad. That said, a lot of real-world shell scripts are bad.

I tend to agree with the overall sentiment; shell isn't a great language. As soon as you start to mix awk (which awk is that, do you need GNU awk?), sed and perhaps a bit of egrep (or grep -E -- are both available? Does it accept only BSD-style parameters?) -- one should consider moving "up".

And for eg: setting up a python package/program -- I'd generally prefer a python script -- hopefully one that handles different file-paths (eg: / vs \ ) and other cross-platform stuff. If you already depend on python, why add dependency on shell?

Re: Ctypes.sh – A foreign function interface for bash

#24

Post by AndyKelley »

e12e wrote: post #23
AndyKelley wrote: post #21

Earlier quoted context omitted.

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

> Can you give a shell script example of something that perl/python/ruby/nodejs can't do? Install perl/python/ruby and nodejs -- requiring just the shell to be installed? Snark aside (it's not only snark - pretty much every system will have something like a posix shell), proper posix portable shell is hard - it's an old gnarly language -- but it's what we've got. And with a bit of discipline and good practices -- it…

> Install perl/python/ruby and nodejs -- requiring just the shell to be installed?

I just meant using any one of them, not all of them. And you can pretty much always rely on python and perl being installed.

But my point is that if you're doing something other than a one-off thing, then it belongs to some project and you're probably going to commit that script to that project's codebase. That means it has to be maintained. Do future you and whoever else has to maintain the project a favor and use one of the popular scripting languages that has reasonable syntax and semantics.

Re: Ctypes.sh – A foreign function interface for bash

#25

Post by adrusi »

AndyKelley wrote: post #22
adrusi wrote: post #20

Earlier quoted context omitted.

In addition to AnimalMuppet's point, shell scripts make no attempt to be cross platform, so you don't get any awkward leaky abstractions when trying to interact with low-level system facilities. The primitives in shell scripts are the primitives of the operating system. The fundamental building blocks of your language are strings and files and processes, which makes it really convenient to work with strings and files…

You can easily invoke platform-specific binaries from within a scripting language. You can even invoke them in a shell if you want. Check out this API: https://docs.python.org/3/library/subprocess.html#subprocess...

Well of course you can. Scripting languages aren't useless stunted toys. The point is that if that's what most of your program is doing, you'll have a lot more success using a language where that functionality is first class.

Re: Ctypes.sh – A foreign function interface for bash

#26

Post by adrusi »

AndyKelley wrote: post #21
adrusi wrote: post #12

Earlier quoted context omitted.

I disagree. Shell scripts are great for automating system tasks — for sysadminy type stuff, including tasks with high complexity with nested loops and conditionals. A scripting language like those you mentioned is great for if you need to run your script on systems you don't control, or you need it to be cross platform, or you'd describe what you're doing as creating software rather than automating tasks. No scriptin…

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

No such example; after all, you can spawn a shell subprocess in any of those.

There's just certain types of tasks where you can more clearly express your intent in a shell script. If, for example, you need to spawn a ton of subprocesses, you can do that in any scripting language, but shell is designed from the ground up to launch subprocesses — it's most basic purpose is to launch programs.

And so while I can't give an example of something that can only be done in a shell script, here's a shell script that I wrote a few weeks ago that would have been a pain in the neck to express in any other language:

    #!/bin/sh
    cd "$(dirname "$0")"
    for test in *.in; do
        test="$(basename "$test" .in)"
        infile="$test.in"
        outfile="$test.out"
        output="$(./whofrom "$infile" 2>&1)"
        expected="$(cat "$outfile")"
        if [ "$output" != "$expected" ]; then
                echo "Failed test $test"
                echo "Expected: $expected"
                echo "Actual:   $output"
                echo
        fi
        if ! valgrind --error-exitcode=1 --leak-check=full ./whofrom "$infile" 2>/dev/null >/dev/null; then
                echo "Failed test $test"
                valgrind -q --leak-check=full ./whofrom "$infile"
                echo
        fi
    done

Re: Ctypes.sh – A foreign function interface for bash

#27

Post by yarrel »

AndyKelley wrote: post #22
adrusi wrote: post #20

Earlier quoted context omitted.

In addition to AnimalMuppet's point, shell scripts make no attempt to be cross platform, so you don't get any awkward leaky abstractions when trying to interact with low-level system facilities. The primitives in shell scripts are the primitives of the operating system. The fundamental building blocks of your language are strings and files and processes, which makes it really convenient to work with strings and files…

You can easily invoke platform-specific binaries from within a scripting language. You can even invoke them in a shell if you want. Check out this API: https://docs.python.org/3/library/subprocess.html#subprocess...

If most of your scripting language code is calls to other programs and to the shell, there's a simpler way...

Re: Ctypes.sh – A foreign function interface for bash

#28

Post by coldtea »

e12e wrote: post #23
AndyKelley wrote: post #21

Earlier quoted context omitted.

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

> Can you give a shell script example of something that perl/python/ruby/nodejs can't do? Install perl/python/ruby and nodejs -- requiring just the shell to be installed? Snark aside (it's not only snark - pretty much every system will have something like a posix shell), proper posix portable shell is hard - it's an old gnarly language -- but it's what we've got. And with a bit of discipline and good practices -- it…

>Install perl/python/ruby and nodejs -- requiring just the shell to be installed?

You can run several of those languages (python for sure, but also perl IIRC) as a shell.

Re: Ctypes.sh – A foreign function interface for bash

#29

Post by viraptor »

adrusi wrote: post #26
AndyKelley wrote: post #21

Earlier quoted context omitted.

Can you give a shell script example of something that perl/python/ruby/nodejs can't do?

No such example; after all, you can spawn a shell subprocess in any of those. There's just certain types of tasks where you can more clearly express your intent in a shell script. If, for example, you need to spawn a ton of subprocesses, you can do that in any scripting language, but shell is designed from the ground up to launch subprocesses — it's most basic purpose is to launch programs. And so while I can't give…

I don't think it's a big pain in python. I imagine ruby wouldn't be terrible either:

    #!/usr/bin/env python

    import sys, os, subprocess, glob 

    os.chdir(os.path.dirname(sys.argv[0]))
    for test in glob.glob("*.in"):
        test = os.path.splitext(test)[0]
        infile = test + ".in"
        outfile = test + ".out"
        output = subprocess.check_output(["./whofrom", infile], stderr=subprocess.STDOUT)
        expected = open(outfile, 'r').read()
        if output != expected:
            print("Failed test", test)
        try:
            devnull = open(os.devnull, 'w')
            subprocess.check_call(["valgrind", "--error-exitcode=1", "--leak-check=full", "./whofrom", infile], stderr=devnull, stdout=devnull)
        except subprocess.CalledProcessError:
            print("Failed test", test)
            subprocess.check_call(["valgrind", "-q", "--leak-check=full", "./whofrom", infile])
(since it's a quick script, I'm ignoring stuff like closing files - they'll get GCd on each iteration anyway)

Apart from the header, the whole script is pretty much the same when comparing line-by-line.

Re: Ctypes.sh – A foreign function interface for bash

#30

Post by jhallenworld »

viraptor wrote: post #29
adrusi wrote: post #26

Earlier quoted context omitted.

No such example; after all, you can spawn a shell subprocess in any of those. There's just certain types of tasks where you can more clearly express your intent in a shell script. If, for example, you need to spawn a ton of subprocesses, you can do that in any scripting language, but shell is designed from the ground up to launch subprocesses — it's most basic purpose is to launch programs. And so while I can't give…

I don't think it's a big pain in python. I imagine ruby wouldn't be terrible either: #!/usr/bin/env python import sys, os, subprocess, glob os.chdir(os.path.dirname(sys.argv[0])) for test in glob.glob("*.in"): test = os.path.splitext(test)[0] infile = test + ".in" outfile = test + ".out" output = subprocess.check_output(["./whofrom", infile], stderr=subprocess.STDOUT) expected = open(outfile, 'r').read() if output !=…

One issue is the problem of indirect documentation, and it comes down to this one line: "import sys, os, subprocess, glob".

How do you find which module you need? Search on the web may be the best answer. In shell at least you have "man -k".

Once I know the module, how do I get its documentation? Here at least there is an answer: import glob; help(glob);

But how good is this documentation? If I do it I get:

    glob(pathname)
    Return a list of paths matching a pathname pattern.
    
    The pattern may contain simple shell-style wildcards a la fnmatch.
Already python is telling me that the "man" documentation is going to be better :-).
Post reply on HN