Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing Safe Shell Scripts (2019)

sipb.mit.edu

Post reply on HN
Print view View original

81–90 of 166 posts

Re: Writing Safe Shell Scripts (2019)

#81

Post by plapetomain »

edgyquant wrote: post #62
Pxtl wrote: post #58

Earlier quoted context omitted.

PowerShell, mostly because PowerShell was designed as a mashup of Bash and C#.... And it's kind of a trainwreck in a lot of ways. It really feels like piping and easy process invocation and compile-time directory awareness wouldn't be massively onerous to add to an existing full-featured programming language so you wouldn't have to sacrifice a good type system and powerful syntax when you want to do scripty things.

Right. Powershell is a shell though not a programming language.

That is false.

Why do you think it is not a programming language?

Re: Writing Safe Shell Scripts (2019)

#82

Post by chubot »

silly_giraffe wrote: post #64
chubot wrote: post #49

Earlier quoted context omitted.

(Oil author here) Yes I think that's a great idea, and I hope to release a "liboil" so that people can do that. Let a thousand flowers bloom, etc. The most likely thing is that Oil is going to be a much better language this year (more expressive, with safety features, and it should be fast), but the interactive UI will still be more bash-like than fish-like: https://www.oilshell.org/blog/2020/01/making-plans.html So…

I want to say that I am continually impressed at your long-term vision and commitment you have taken to the Oil project. Your posts are a treat to read, and the care you have given to the problem is appreciated. We need to stop paving cow-paths, and discard historical baggage if we are ever going to build robust systems.

Thank you!

Although I have to say I do feel the tradeoff right now between having a strong design / globally consistent code vs. allowing local variation / a lot of people to contribute quickly. That is, allow people to get in, add their useful patch, their useful bit of knowledge, and get out, without caring at all about the rest of the program.

I think Linux, git, and GNU are the prototypical examples of the latter. Those projects are too big for one person, so the code is set up in a way to allow hacking locally. They're also arguably a mess. Anyone who's ever written against the container APIs in Linux knows what I mean (namespaces, cgroups, etc.)

The git UI is another famous example, i.e. the plethora of commands and flags that make little sense. Empirically it seems that you can move faster if you disregard consistency and coherence.

The shell language of course grew like that over 50 years. Bourne shell is pretty consistent, but one thing I learned is that ksh added a lot of bash misfeatures (they're ksh-isms not bash-isms), and it's pretty bad.

But shell is actually a significantly smaller problem than git (after all bash is maintained more or less by one person). So it should be possible to fix it and redesign it. Unfortunately I'm not sure if the lessons generalize to bigger projects, but I would like it if they do.

-----

On a more practical note, feel free to send more people this way if you want to see the project succeed :) The design is there, and it will hold up, but it needs a bit more manpower, especially for things like the interactive shell.

Re: Writing Safe Shell Scripts (2019)

#84

Post by bproven »

pfranz wrote: post #26
lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

I've primarily used Python 2 for 10+ years and I often find cases where shell scripts are preferable. The major differentiator is usually "shelling out" in Python kind of sucks. It's verbose, output collection and error handling suck, and escaping can be miserable. I often will reimplement things in pure Python if I have the time. A recent example was I needed to tar+split large files. `tar cf - -C / $filename | spli…

I agree, but this lib does make it easier: https://github.com/amoffat/sh

Re: Writing Safe Shell Scripts (2019)

#85

Post by chubot »

arendtio wrote: post #75

Some time ago I started to doubt if using 'set -e' is a good idea. I mean, if it would work as you expect it to, it certainly is a good idea to exit a script as soon as something fails. But sadly not all implementations behave similarly [1] and if you call a function from within a condition, 'set -e' gets deactivated/doesn't work. For illustration, take a look at the following example: #!/bin/bash foo() { set -e fals…

Yeah there's no good solution for this in shell -- you're damned if you do, and damned if you don't.

bash has inherit_errexit to fix a longstanding bug, but even that's not enough.

Oil has shopt -s strict_errexit and more_errexit that fixes even more.

https://github.com/oilshell/oil/issues/476

I'm still looking for feedback and there's probably one more thing I want to change.

Although the fundamental problem is that errors (success/fail) and boolean values (true/false) are conflated in the status code, so it's a tough problem.

In Oil you will be encouraged to use expressions for booleans, not statements, e.g.

    ls /    # success/fail
    if (x > 0) { }  # true/false
rather than

    ls /
    if test $x -lt 0; then ...

Re: Writing Safe Shell Scripts (2019)

#86

Post by seorphates »

seorphates wrote: post #63

Not to be that guy but this is mostly garbage. I suggest simply paying attention and testing. Switches can be a useful part of the tool but they can break things. It doesn't make things better rather it alters behaviors of the shell in sometimes unintended ways. Try some loops with some -eo. If you have a pipe that fails your script you're just ripping the heart and potential out of your script. I think you might jus…

Ok, "garbage" may have been harsh but how about a fun example? Try this with and without -e

  #!/usr/bin/env bash

  bell=`tput bel`
  tock='Blastoff!'

  do_ring()
  {
    if [ "$1" ]; then # true
      echo -n $bell; sleep 0.1
      echo -n $bell; sleep 0.1
      echo -n $bell; sleep 0.1
      echo $tock
    else
      echo -n $bell; sleep 0.1
    fi
  }

  i=3
  while [ "$i" -ge "0" ];
  do
    if [ $i = 0 ]; then
      sleep 0.5
      echo ok
    else
      echo $i $bell
      sleep 0.5
    fi
    i=`expr $i - 1`
  done

  do_ring $1 # do something different if arg
  sleep 1
  echo neat.
Semantic qualities aside there is something here that breaks with -e and if you're used to using -e by default you might be baffled and think shell sucks, for example.

Re: Writing Safe Shell Scripts (2019)

#87

Post by dahart »

lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

I love python, and half my ‘shell’ scripts are python, but the mere fact that I have to import something to exec something or read a file means that bash is easier to build and test one line at a time. And good bash pipelined one-liners are a reason I’ll never write all of my scripts in python. It’d be hard for me to calculate how often I use constructs like ‘grep | sort | uniq | cut’. That takes a lot of python code.

Re: Writing Safe Shell Scripts (2019)

#88

Post by ori_b »

lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

Practically speaking, my shell scripts tend to be something like a tenth the size of the equivalent python

Re: Writing Safe Shell Scripts (2019)

#89

Post by dahart »

vbernat wrote: post #13

Since it's bash-specific, I think people should use zsh instead. No need to quote variables as by default, splitting is not done. And you get access to arrays and associative arrays.

I love zsh’s variable & filename expansion features! But I switched to bash after using zsh for a long time because zsh is less standard, not installed by default, can be a bit of a pain when doing lots of ssh work or on machines I don’t control. Bash is always there.

Zsh is also really heavy, or it was last time I checked. Larger binary & slower startup time than bash, and lots of features I don’t use. Modules, calendar, tcp, ftp client ... that’s a lot of stuff in the shell that is arguably better put into separate executables.

Re: Writing Safe Shell Scripts (2019)

#90

Post by clarry »

seorphates wrote: post #86
seorphates wrote: post #63

Not to be that guy but this is mostly garbage. I suggest simply paying attention and testing. Switches can be a useful part of the tool but they can break things. It doesn't make things better rather it alters behaviors of the shell in sometimes unintended ways. Try some loops with some -eo. If you have a pipe that fails your script you're just ripping the heart and potential out of your script. I think you might jus…

Ok, "garbage" may have been harsh but how about a fun example? Try this with and without -e #!/usr/bin/env bash bell=`tput bel` tock='Blastoff!' do_ring() { if [ "$1" ]; then # true echo -n $bell; sleep 0.1 echo -n $bell; sleep 0.1 echo -n $bell; sleep 0.1 echo $tock else echo -n $bell; sleep 0.1 fi } i=3 while [ "$i" -ge "0" ]; do if [ $i = 0 ]; then sleep 0.5 echo ok else echo $i $bell sleep 0.5 fi i=`expr $i - 1`…

That's why nobody uses expr for arithmetic. But yes, it is annoying that -e breaks some habits that work fine without it.
Post reply on HN