Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debugging Bash Like a Sire (2023)

blog.brujordet.no

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Debugging Bash Like a Sire (2023)

#31

Post by thaumasiotes »

imglorp wrote: post #28
latexr wrote: post #13

Earlier quoted context omitted.

As someone who has been writing shell scripts for a few decades (though not as long as you), I’d instead recommend “learn what your tools are appropriate for and use them that way”. There are plenty of cases where shell scripts are the right tool for the job. I can’t even tell how many times I’ve seen multi-line Python scripts which could instead have been a shell one-liner. Shorter and faster. I have also written sh…

I'll put a point on the "it depends" bit. If you have a standard-ish environment, you'll have an array of Unix tools to compose together, which is what a shell is best at. Even a minimal image like busybox will have enough to do serious work. Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. As soon as you say "switch to (favorite decent scripting envir…

> Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner.

> As soon as you say "switch to (favorite decent scripting environment)", you're committing to (a) many megs of its base install, (b) its package management system, (c) whatever domain packages you need for $work, and (d) all the attendant dependency hells that brings along.

OK, but isn't jq just an example of a favorite scripting environment with a multi-meg install and a dependency system? What are you doing that's different from what you're advising everyone else not to do?

> Golfing in a scripting environment is composing a bunch of builtin operations.

Neither curl nor jq is a builtin operation.

Re: Debugging Bash Like a Sire (2023)

#32

Post by sgarland »

kajkojednojajko wrote: post #9
mjd wrote: post #3

I've been writing scripts in Bourne shell since the 1980s and in Bash since whenever it came along, and I feel like the most important thing I've learned about it is: don't. Sure, it can be done, and even done well, but why? There are better languages. Every time I write a shell script that grows to more than about 20 lines I curse myself for not having written it in Python. The longer I have waited before throwing i…

> the most important thing I've learned about [bash] is: don't. Sure, it can be done, and even done well, but why? There are better languages. This. Bash gives you all the tools to dig a hole and none to climb out. It's quick and easy to copy commands from your terminal to a file, and it beats not saving them at all. Support for digging: once you have a shell script, adding one more line conditioned on some env var i…

> no modules

Ish. You can source whatever files you want, so if you split up your functions into logical directories / files, you can get modules (-ish).

> no tests

BATS [0].

[0]: https://github.com/bats-core/bats-core

> I've successfully refactored messy python code in the past, but with bash I've had no idea where to even start.

I say this with all kindness: you probably need to know more bash before you can safely refactor it. It is a very pointy and unforgiving language.

Re: Debugging Bash Like a Sire (2023)

#33

Post by thaumasiotes »

kajkojednojajko wrote: post #9
mjd wrote: post #3

I've been writing scripts in Bourne shell since the 1980s and in Bash since whenever it came along, and I feel like the most important thing I've learned about it is: don't. Sure, it can be done, and even done well, but why? There are better languages. Every time I write a shell script that grows to more than about 20 lines I curse myself for not having written it in Python. The longer I have waited before throwing i…

> the most important thing I've learned about [bash] is: don't. Sure, it can be done, and even done well, but why? There are better languages. This. Bash gives you all the tools to dig a hole and none to climb out. It's quick and easy to copy commands from your terminal to a file, and it beats not saving them at all. Support for digging: once you have a shell script, adding one more line conditioned on some env var i…

I wrote a powershell script to run an ffmpeg workflow. I'm confident that this was a better idea than either of the other two approaches that you seem to be advocating for:

(a) instead of writing a shell script to operate a shell-operated tool, write a python script with a bunch of os.system('shell out') commands.

(b) instead of just invoking ffmpeg to do the things you want done, install an ffmpeg development library, and call the functions that ffmpeg itself calls to do those things.

What would be the argument for either of those?

Re: Debugging Bash Like a Sire (2023)

#34

Post by latexr »

thaumasiotes wrote: post #31
imglorp wrote: post #28

Earlier quoted context omitted.

I'll put a point on the "it depends" bit. If you have a standard-ish environment, you'll have an array of Unix tools to compose together, which is what a shell is best at. Even a minimal image like busybox will have enough to do serious work. Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. As soon as you say "switch to (favorite decent scripting envir…

> Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. > As soon as you say "switch to (favorite decent scripting environment)", you're committing to (a) many megs of its base install, (b) its package management system, (c) whatever domain packages you need for $work, and (d) all the attendant dependency hells that brings along. OK, but isn't jq just an ex…

> OK, but isn't jq just an example of a favorite scripting environment with a multi-meg install and a dependency system?

No? jq is a single binary a little over half a MB with no runtime dependencies. You can simply download it and use it. And you only need that if it’s not already included in whatever system you’re using, which it likely is. It even comes with macOS these days, which is more than what you can say for Python.

https://news.ycombinator.com/item?id=44408432

> Neither curl nor jq is a builtin operation.

Pretty sure they meant it as in builtin with the system, not the language. As per their first paragraph:

> Even a minimal image like busybox will have enough to do serious work.

Re: Debugging Bash Like a Sire (2023)

#35

Post by sgarland »

hn92726819 wrote: post #22
mjd wrote: post #21

Earlier quoted context omitted.

That's just the problem! It is better at the task. Until it isn't, and "isn't" comes much too soon.

I've seen this sentiment a lot here. "Once shell is >n lines, port to python". My experience has been different. Maybe half of the scripts I write are better off in python while the other half are exponentially longer in python than bash. For example, anything to do with json can be done in 1 line of readable jq, while it could be 1, 5, or 20 lines in python depending on the problem. I'd just like to put that out the…

Same. It’s mostly because if I have a shell script, while I’ll add some comments for tricky bits if needed, and maybe a `-h` function, that’s about it. In Python, though, I use the language’s features to make it as readable and safe as possible. Types, docstrings, argparse, etc. My thinking is that if I’m going to take the time to use a “proper” language, then I should make it bulletproof, otherwise I’d just stick with shell.

My personal decision matrix for when to switch to Python usually involves the relative comfort of the rest of my team in both languages, the likelihood that future maintenance or development of the script will be necessary, and whether or not I’m dealing with inputs may change (e.g. API responses, since sadly versioning isn’t always a guarantee).

Re: Debugging Bash Like a Sire (2023)

#36

Post by thaumasiotes »

latexr wrote: post #34
thaumasiotes wrote: post #31

Earlier quoted context omitted.

> Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. > As soon as you say "switch to (favorite decent scripting environment)", you're committing to (a) many megs of its base install, (b) its package management system, (c) whatever domain packages you need for $work, and (d) all the attendant dependency hells that brings along. OK, but isn't jq just an ex…

> OK, but isn't jq just an example of a favorite scripting environment with a multi-meg install and a dependency system? No? jq is a single binary a little over half a MB with no runtime dependencies. You can simply download it and use it. And you only need that if it’s not already included in whatever system you’re using, which it likely is. It even comes with macOS these days, which is more than what you can say fo…

> No? jq is a single binary a little over half a MB with no runtime dependencies.

I just downloaded it to see what size it was, and it's 2200 KB.

> Pretty sure they meant it as in builtin with the system, not the language. As per their first paragraph:

>> Even a minimal image like busybox will have enough to do serious work.

Busybox doesn't include curl or jq. They aren't builtins by any standard.

This becomes obvious every time you set up a new machine and try to curl something, and then realize you have to install curl.

Re: Debugging Bash Like a Sire (2023)

#37

Post by t43562 »

thaumasiotes wrote: post #31
imglorp wrote: post #28

Earlier quoted context omitted.

I'll put a point on the "it depends" bit. If you have a standard-ish environment, you'll have an array of Unix tools to compose together, which is what a shell is best at. Even a minimal image like busybox will have enough to do serious work. Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. As soon as you say "switch to (favorite decent scripting envir…

> Golfing in shell can be a pipeline of tools: lately "curl | jq | awk" does a lot of lifting for me in a one-liner. > As soon as you say "switch to (favorite decent scripting environment)", you're committing to (a) many megs of its base install, (b) its package management system, (c) whatever domain packages you need for $work, and (d) all the attendant dependency hells that brings along. OK, but isn't jq just an ex…

Fair enough point but for many years I wasn't aware of what bash COULD do. I mean one should get to learn more about [[]] and how it does regexps and while read loops:

ls *.txt | { while read FILENAME; do to $FILENAME; done; }

and so on. Once you know, you can get a lot done on e.g. a docker image, without having to install lots of other things first.

Re: Debugging Bash Like a Sire (2023)

#38

Post by darkwater »

cerved wrote: post #20
jelder wrote: post #19

Earlier quoted context omitted.

This is an excellent point about pipes. There seems to be no other language which lets you stitch together pipes like Bash does. It's incredibly powerful, and worth putting up with all of Bash's warts. Thanks for fzf, by the way. Always one of the first things I install in a new environment.

They are not the author of fzf

Tbh the sentence + the link URL also tricked me into thinking that initially.

Re: Debugging Bash Like a Sire (2023)

#40

Post by bmoyles »

You can also skip the subshell invocation of date by using %(fmt)T from bash's printf:

  %(fmt)T -output the date-time string resulting from using FMT as a format string for strftime(3)
The man page provides a bit more detail:

  %(datefmt)T causes printf to output the date-time string resulting from using datefmt as a format string for strftime(3).  The corresponding argument  is  an  integer
  representing  the  number of seconds since the epoch.  Two special argument values may be used: -1 represents the current time, and -2 represents the time
  the shell was invoked.  If no argument is specified, conversion behaves as if -1 had been given.  This is an exception to the usual printf behavior.
With that,

    timestamp=$(date +'%y.%m.%d %H:%M:%S')
becomes

    printf -v timestamp '%(%y.%m.%d %H:%M:%S)T' -1
Post reply on HN