Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe ways to do things in bash

github.com

Post reply on HN
Print view View original

241–250 of 255 posts

Re: Safe ways to do things in bash

#241

Post by Groxx »

h1d wrote: post #154
davedx wrote: post #152

Earlier quoted context omitted.

Yes, but which one, and which version? This is especially painful with Python ;)

What's painful about Python but not so for others? Writing anything complicated in bash/zsh is painful enough I wouldn't even think about it.

2 vs 3 is not a small change. and tools for helping you do things (like a 3+2 compatibility lib) get installed with pip... which is horrifyingly fragile. `pip install your-lib` -> you have no idea if you have the dependencies you require, because pip doesn't behave rationally. it often Just Works™, but when it doesn't there's no help and a massive minefield of potential problems.

Re: Safe ways to do things in bash

#242

Post by [deleted] »

JdeBP wrote: post #197
jwilk wrote: post #149

Earlier quoted context omitted.

I'm all for "command -v", but do you know which OSes don't have "which" by default?

A straw man question. That's not the problem with which. * https://github.com/koalaman/shellcheck/wiki/SC2230 * https://unix.stackexchange.com/questions/85249/

[deleted]

Re: Safe ways to do things in bash

#243

Post by freedomben »

emmelaich wrote: post #147
freedomben wrote: post #69

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

I probably have a similar history but I've come to the belief that you should not use Bash if you're doing anything fancy or long (>10 lines). Just use Perl, Python or Ruby. One or all of them is installed on every machine you are likely to use.

Except for perl, python and ruby are rarely on container images and AMIs I interact with unless they are needed for the application itself to minimize attack surfaces and vectors. Perl is installed sometimes due to dependent packages that themselves depend on perl, but that's becoming much less true as time goes on. With Docker it's also very easy to keep your image super minimal. Most of the time we don't even have bash installed (just sh).

Re: Safe ways to do things in bash

#244

Post by freedomben »

brazzledazzle wrote: post #167
h1d wrote: post #155

Earlier quoted context omitted.

It's time we need a better shell than bash instead of thinking it's great after bleeding with it for years and gets used to it.

I’d vote for PowerShell in a heartbeat. It passes real objects instead of strings. It’s cross platform and open source. It’s imperative but borrows some functional concepts.

I really wonder if PowerShell wouldn't have been much bigger had it been cross-platform from the start. Definitely better late than never (:clap: Microsoft) but at least in my sphere we never even thought seriously about PowerShell because we had too many linux machines in our environment.

Re: Safe ways to do things in bash

#245

Post by freedomben »

zippitydoodah68 wrote: post #238
freedomben wrote: post #121

Earlier quoted context omitted.

I personally consider awk, sed, grep, cut, etc. to be "part" of bash. I don't think I ever write a script without invoking those venerable tools at least once. I often have many pipes, such as some_command \ | grep -E 'some.*regex$' \ | sed -e 's/erase_text//g' \ | sed -e 's/erase_more//g' \ | awk '{ print $2 }' \ | cut -d ':' -f 1 I used to reach for Perl one-liners a lot, and still do sometimes when I need a gross…

These types of pipelines are why bash and the traditional tools are so maligned. Expert knowledge of sed|(g)awk|bash tools is necessary otherwise you end up with companies telling you that you can't use them. some_command | gawk ' /some.*regex$/ {gsub(erase_text,"");gsub(erase_more,"");split($2,a,":"); print (length(a[1]) ? a[1] : "STRING ERROR")}'

I'll definitely agree that it can get out of hand. If it's a script/tool that will have many eyes on it and people that need to understand it, bash tools probably aren't the best way to go. I usually turn to ruby in those cases. But even for one off commands, I find myself using pipes like that all the time and I'm the only one that will ever see it, so if it's greek it's no problem :-)

Re: Safe ways to do things in bash

#246

Post by collinmanderson »

h1d wrote: post #153
freedomben wrote: post #69

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

I think it's time people start using something better than bash/zsh that is decades old, like fish or even come up with a more modern shell. Even by looking at these examples, you see it has less verbosity like "then" and "do", you can reference arguments as $argv instead of cryptic $@ and exit status code as $status instead of $? which is confusing with $! and the likes. https://blog.codeship.com/lets-talk-about-she…

Doesn’t the original article say fish isn’t safe?

> Fish is easier to use correctly, but lacks a safe mode. Prototyping in fish is therefore a good idea, provided that you know how to translate correctly from fish to bash.

Re: Safe ways to do things in bash

#247

Post by posix_me_less »

Annatar wrote: post #240
posix_me_less wrote: post #174

Earlier quoted context omitted.

> "This is how one can tell a rookie just learning to program in shell: usage of $() syntax is limited to Bourne family of shells which implement that particular POSIX specification aspect. Backticks, on the other hand, although they incur a performance penalty since they spawn a subshell, make one's code instantly portable across all UNIX-like operating systems and even across disparate shell families, as they work…

“Long live the GNU/Linux hegemony and monoculture, the only truth and true religion”. Lovely.

Long live the inventiveness and free spirit of contributors who brings us useful improvements and progress to what would otherwise be a cumbersome legacy computer interface.

Re: Safe ways to do things in bash

#248

Post by raboukhalil »

freedomben wrote: post #69

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

>> I'll grant you tho that arrays are still nasty ;-)

If you're interested, here's an article I wrote recently that attempts to explain the method in the madness: https://medium.com/p/the-weird-wondrous-world-of-bash-arrays...

Re: Safe ways to do things in bash

#249

Post by Annatar »

posix_me_less wrote: post #247
Annatar wrote: post #240

Earlier quoted context omitted.

“Long live the GNU/Linux hegemony and monoculture, the only truth and true religion”. Lovely.

Long live the inventiveness and free spirit of contributors who brings us useful improvements and progress to what would otherwise be a cumbersome legacy computer interface.

Which interface? Are you sure you could not have made a broader generalization and a more nondescript statement?

Re: Safe ways to do things in bash

#250

Post by scbrg »

meow_mix wrote: post #212
scbrg wrote: post #160

Earlier quoted context omitted.

Because foo | awk '/bar/ {print $3}' is more clear than import subprocess foo = subprocess.Popen(['foo'], stdout=subprocess.PIPE) for line in foo.stdout: if 'bar' in line: try: print(line.split()[2]) except IndexError: print('') Sometimes shell scripts are more clear. Especially for tasks that involve running lots of external commands.

Nobody doubts that pipes and languages like awk are great for one liners, but I think that's a little besides the point of this post, which is advocating for things like the use of bash arrays: ``` array=( a b ) array+=(c) if [ ${#array[@]} -gt 0 ]; then rm -- "${array[@]}" fi ``` ``` array = [a, b] array 0 ``` There's also nobody stopping you from using text processing tools like awk and sed, or bash one liners in r…

You're moving the goalposts! Previously it was "anything but install scripts". Now it's "logic and arrays".

I actually think we agree with each other, but we express it differently. I never write bash scripts, I write POSIX shell, so all the array juggling of bash is something I never deal with. As you say, by the time you need arrays, you should have switched languages already.

That said, I think there's a fairly large domain of problems - apart from install scripts - that are better solved with shell scripts, because of their clarity. Anything which relies on invokation of lots of other tools, and in particular problems that fit the pipe model (take output from this tool, extract interesting bits from it, and feed it to that tool, etc). And this I say as an otherwise almost slightly fanatical Pythonista :-)

Post reply on HN