How to do things safely in Bash (2018) • Hacker News Board

Re: How to do things safely in Bash (2018)

#41

Post by chubot » Sat, Apr 03, 2021, 9:52 PM UTC

jrockway wrote: post #26

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

Same. I feel like I spend 45 minutes every time I write a shell script to determine simple things like "is this environment variable set". Stack Overflow always has 100 answers on how to do this, and every one has someone replying "well that doesn't work if..." In general, after 20 years of using Unix, I'm starting to sour on the "everything is a string" and "streams are just newline separated plain-text records". Th…

Have you tried any of the shells here?

https://github.com/oilshell/oil/wiki/Alternative-Shells

Discussed recently: https://news.ycombinator.com/item?id=26121592

A lot of them are "fundamentally reimagining" shell -- actually I'd say MOST of them are; whether that's good or bad depends on the user's POV.

Oil is reimagining shell, but also providing a graceful upgrade path. Out of all the shells I'd say it's most focused on the fundamental language and runtime, and less on the interactive issues (right now).

Re: How to do things safely in Bash (2018)

#42

Post by chubot » Sat, Apr 03, 2021, 9:53 PM UTC

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

Shouldn't scripts over 100 lines be rewritten in Python or Ruby?

http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#sh...

tl;dr I think the main skill that is missing is being able to write a Python script that fits well within a shell script. The shell script often invokes tools written in other languages like C, Rust, JavaScript, etc. (most of which you didn't write)

Good book on this: https://www.amazon.com/UNIX-Programming-Addison-Wesley-Profe...

Online for free: http://www.catb.org/esr/writings/taoup/html/

Re: How to do things safely in Bash (2018)

#43

Post by fillest » Sat, Apr 03, 2021, 10:01 PM UTC

kubanczyk wrote: post #40

blabitty wrote: post #18

Earlier quoted context omitted.

I wonder too. Things like file and directory manipulations are way more complicated in native Python and I find the exceptions harder to troubleshoot than native commands. I'm not even going to get into pythons version and dependency hell, suffice it to say I have more confidence in a standard version of bash and standard utilities generally being present on a modern Linux system. What I've seen from pythonistas as r…

Here is a novel idea. I am allowed to call 'mv' or 'cp' binaries from bash. I am also allowed to call 'mv' or 'cp' binaries from python with subprocess! (Edit: bad example binaries, 'df' or 'tar' would be probably better.) I must choose not to install bash libraries for a bash script beyond builtins. I can choose not to install python libraries beyond builtins. It becomes a really simple tradeoff. Nobody forces you t…

It is OK to call whatever one needs actually - just minimize the usage of Bash language features (Python provides most of them). It is not complicated and requires no dependencies, here's a self-contained helper: https://gist.github.com/fillest/8d64f8fa0cdb1745bfc9c683cf39...

Re: How to do things safely in Bash (2018)

#44

Post by Supersaiyan_IV » Sat, Apr 03, 2021, 10:37 PM UTC

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

At least shell follows POSIX standard which is more than one can say about Python. Have witnessed self-proclaimed "shell replacing" python scripts invoking commands, and never checking the exit code. Debugging hell.

Re: How to do things safely in Bash (2018)

#45

Post by kaba0 » Sat, Apr 03, 2021, 10:49 PM UTC

maxrev17 wrote: post #19

See the same debate ensue each time with bash - use python, they shout. Perhaps the overriding factor here is familiarity? Past looping and some basic flow control I'm guessing most people run out of being comfortable in bash and can get things done quicker in their fave Lang?

It’s not about being comfortable, but about the million ways your shell script will fail. A file has a space in the name? Good chance your script failed. No file founs? Same. Multiple files found after expansion? Same.

Re: How to do things safely in Bash (2018)

#46

Post by kaba0 » Sat, Apr 03, 2021, 10:52 PM UTC

effie wrote: post #29

macintux wrote: post #16

Earlier quoted context omitted.

How many strongly typed languages are: + Installed everywhere + Even within 20% as concise as bash

Indeed. Python does not win over bash in either category, and it is slower. I wish there was a super-fast shell-like scripting language everywhere that could serve as a general programming language. Like Perl, but simpler rules and readable syntax. But there is nothing. Who would have thought, I may have to learn Perl eventually.

How is it slower/matter at all? Goddamn bash calls an external process in if branches, so if a bash script is enough for a given process, than running python inside a vm inside a vm inside a vm will be still fast enough. Scripting doesn’t require performance.

Re: How to do things safely in Bash (2018)

#47

Post by rectang » Sat, Apr 03, 2021, 11:05 PM UTC

marcosdumay wrote: post #30

stormbrew wrote: post #14

Earlier quoted context omitted.

This, and I mean very specifically this with python always as the go-to replacement, comes up every single time anyone even utters the words "shell" or "bash" and every time all I can think is that the people saying it aren't writing the same kinds of shell scripts I ever see, use, or wind up writing. And every time I've ever seen something that's allegedly "a shell script replaced with a python script" it's way more…

Hum, no those people are probably writing the same kind of script you do. Their Python versions are indeed longer and more verbose. I always recommend Python anyway if you care about edge cases (what is not always). That longer script handles them on the obvious way, while the short and more readable shell script does something absolutely crazy every time a detail is different from planed. And if you try to correctly…

Which of these is simpler?

Rust:

    fn add(a: i32, b: i32) -> i32 {
        a + b
    }

Python:

    def add(a, b):
       return a + b

Arguably the Python function is "simpler" because it occupies fewer characters. And yet, the Rust function is more tightly specified; it does less. It can't throw exceptions, unlike the Python function. Invalid programs where you pass the wrong type to the Rust function won't even compile, whereas the Python function will blow up at runtime.

I figure that these supposedly "simpler" Bash programs are not simple at all.

Re: How to do things safely in Bash (2018)

#48

Post by bobbylarrybobby » Sat, Apr 03, 2021, 11:14 PM UTC

enriquto wrote: post #27

nas wrote: post #6

The fact that you have to use quoting nearly everywhere is a design flaw in the Borne shell. Some shells, like Plan 9's rc, for example, don't expand after variable substantiation. They have an operator to call if you want to explicitly force expansion. That's so much cleaner and less error prone.

> The fact that you have to use quoting nearly everywhere is a design flaw in the Borne shell I disagree. This can be considered a design flaw in other places, like filesystems that allow filenames with spaces and other idiotic complexity-inducing things.

Filenames with spaces only introduce complexity because bash was poorly designed. If you could treat strings with spaces the same as strings without spaces — as you can pretty much everywhere but the shell — then there wouldn’t be any additional complexity at all.

Re: How to do things safely in Bash (2018)

#49

Post by macromagnon » Sat, Apr 03, 2021, 11:26 PM UTC

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

I like bash because it's easy to connect different data together. I can output a column from a sql query to a file and easily write a for loop that goes over the data and use it.

I work mostly in payment related code, so most often I use bash to hot fix in production. Like if we need to refund certain people, I'll convert the refund api call to a curl call, get the data from sql and then just run a simple loop.

Re: How to do things safely in Bash (2018)

#50

Post by [deleted] » Sat, Apr 03, 2021, 11:29 PM UTC

jrockway wrote: post #26

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

Same. I feel like I spend 45 minutes every time I write a shell script to determine simple things like "is this environment variable set". Stack Overflow always has 100 answers on how to do this, and every one has someone replying "well that doesn't work if..." In general, after 20 years of using Unix, I'm starting to sour on the "everything is a string" and "streams are just newline separated plain-text records". Th…

[deleted]