Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bash Hackers Wiki

wiki.bash-hackers.org

Post reply on HN
Print view View original

61–70 of 89 posts

Re: The Bash Hackers Wiki

#61

Post by dijit »

wodenokoto wrote: post #50

I clicked the config article and it reminded me of 2 major pet peeves: 1) No good way to store config values, nor ones that can be shared with other languages. I might have a set of python applications running in the cloud and a shell script to deploy them, and guess what, I'd like them to share the same config, so I don't have to keep to update resource names in 2 places. 2) No simple way of templating files. A scri…

Maybe I’m misunderstand the points here but shouldn’t point 1 be covered by environment variables mostly?

And for point 2, basically the same thing, you can set variables to have a default value and then try to read the environment.

export variable=${variable:-“some_default”}

Re: The Bash Hackers Wiki

#62

Post by vaingloriole »

epr wrote: post #48
ekianjo wrote: post #47

Earlier quoted context omitted.

Python needs to be installed, maintained, and every single library will have its list of dependencies (along with breaking API changes once in a while). There's a reason Bash exists: it's everywhere by default and you know it will still work as it works now in x years.

I'm sorry, but for the type of work he is suggesting (bash script equivalent), you don't need anything besides the standard library. I cannot think of a single system I have used in recent decades that didn't come with python installed by default. For all intents and purposes, Python is as ubiquitous as bash at this point.

Balderdash and hokum. z=1 ; while test ${z} -lt 4 ; do a=${RANDOM:1:3} ; dd if=/dev/urandom count=12 bs=4 of=${a} && b=$(cat ${a}) && echo ${b:$(expr 1 + ${RANDOM} % 4):12} ; ((z++)); done

You have one tool that you know well. I have thousands of tools that I've learned daily and I wrap with a bash script..except when I need python/tcl or C. Most of what I do in python/tcl/C is truly complicated, needs to be fast or use a data structure other than an array: as it should be. Most of what I do in the shell is as simple as the userspace tools and bash will let it be. Write the one liner above in python without importing a library.

Re: The Bash Hackers Wiki

#63

Post by ofrzeta »

vaingloriole wrote: post #62
epr wrote: post #48

Earlier quoted context omitted.

I'm sorry, but for the type of work he is suggesting (bash script equivalent), you don't need anything besides the standard library. I cannot think of a single system I have used in recent decades that didn't come with python installed by default. For all intents and purposes, Python is as ubiquitous as bash at this point.

Balderdash and hokum. z=1 ; while test ${z} -lt 4 ; do a=${RANDOM:1:3} ; dd if=/dev/urandom count=12 bs=4 of=${a} && b=$(cat ${a}) && echo ${b:$(expr 1 + ${RANDOM} % 4):12} ; ((z++)); done You have one tool that you know well. I have thousands of tools that I've learned daily and I wrap with a bash script..except when I need python/tcl or C. Most of what I do in python/tcl/C is truly complicated, needs to be fast or…

What's the point of not importing a library/module? It's not like cheating or something.

Re: The Bash Hackers Wiki

#64

Post by vaingloriole »

ShorsHammer wrote: post #43
heavyset_go wrote: post #33

Earlier quoted context omitted.

I'd be as concerned with someone not opting to use what the documentation recommends[1] in lieu of the stdlib, as I would someone who puts together an awkward Bash script to parse JSON instead of just installing the 50kb package for jq. > The Requests package is recommended for a higher-level HTTP client interface. [1] https://docs.python.org/3/library/urllib.request.html

It's a lot easier to hire people than fire them friend. The documentation recommends nothing of the sort, it's a direction for consumers. We all live in varied and different worlds, how many varied server environments have you been responsible for all at once? Is it really beyond your amazing programming skills to use the tools at hand in every circumstance? Perhaps it's a chance to demonstrate yet another 3rd party…

Would never use python in lieu of curl/jq for json handling unless I am writing something more complicated than a shell script. catch 22.

Re: The Bash Hackers Wiki

#65

Post by vaingloriole »

saagarjha wrote: post #39
yesenadam wrote: post #30

Earlier quoted context omitted.

Also true of Python, but I almost never hear people say that. Strange.

Python's a bit faster than Bash.

For what? Is it faster than (g)awk? How about sed? How about dd? How about any of the thousands of tools in userspace that you wrap in bash and are faster than python at the same task? Use the right tool is being lost in the 'you are too stupid to make a good decision, do it this way' generation.

Re: The Bash Hackers Wiki

#66

Post by saagarjha »

vaingloriole wrote: post #65
saagarjha wrote: post #39

Earlier quoted context omitted.

Python's a bit faster than Bash.

For what? Is it faster than (g)awk? How about sed? How about dd? How about any of the thousands of tools in userspace that you wrap in bash and are faster than python at the same task? Use the right tool is being lost in the 'you are too stupid to make a good decision, do it this way' generation.

It depends, of course! But usually the issue with shell scripts is that they create many new processes, which has high overhead. In the instances you're spending more time in dd than bash, in Python you're probably inside an in-process memcpy.

Re: The Bash Hackers Wiki

#67

Post by hannob »

ofrzeta wrote: post #6
dijit wrote: post #3

Bash was the first scripting language I ever got "good" with. (I put "good" in quotes because while I could get basically anything done, I promise that it was not good looking). However, I feel like it left me a bit brain damaged, I still think in terms of quotation quirks and "sub-shells" even in proper languages such as python or rust. It also left me a bit weird when thinking about concurrency, because I never con…

I've come to use more and more Python for tasks I would have used Shell in the past. Because of weird syntax with anything but simple variables or endless fiddling with quoting. You can get a lot done quite easily and concise in Python 3, for instance with the pathlib module that has methods for recursive globbing and stuff like that. https://docs.python.org/3.7/library/pathlib.html

Actually I have recently analyzed some python code that does some shell calls and if I can eliminate them with pure python.

It was quite frustrating. A lot of stuff that the shell has simple functionality for can't be done easily in python.

Examples:

* Call to chown -R. Sure, python can change file owners (os.chown()). But it has no parameter to do this recursively. You need to build a loop around it over some file iteration function.

* Call to setfacl. Can't be done with the python standard library, needs pylibacl and you can't rely on it being installed.

* Call to killall -HUP [processname]. You can send kill signals with python, but only to pids, not to process names. You'd have to parse the system process list or something like that in order to simulate.

In all cases I decided to keep the shell call, it just wasn't worth it converting that to less readable python code.

Re: The Bash Hackers Wiki

#68

Post by vaingloriole »

Too wrote: post #51
_jal wrote: post #14

Earlier quoted context omitted.

Python's major glaring problem as a systems language is dealing with processes. It is just verbose, ticky and annoying. I have my other issues with it as a systems language that are more idiosyncratic, but for replacing bash, process management is just way too much hassle.

99.9% of all process management can be done with oneliner subprocess.check_output. Not verbose at all and much safer. Biggest mistake people do is bringing bash idioms into python, like trying to pipe the output through head/cut/tail/grep/sed instead of just using string operators like endswith/in/startswith/re. If you do that then process managment will be more verbose, but it is in bash also unless you ignore decen…

Do you really think the python popen() workalike with optional shell inclusion is better than running in a shell? More secure perhaps? Easier to read? I forced a developer from my team who would not stop using subprocess calls (including $SHELL) in random new python scripts with hard coded arguments instead of writing a single shell script that could be reused. subprocess is a tarpit and is more misused than any other python function.

Re: The Bash Hackers Wiki

#69

Post by mhd »

ekianjo wrote: post #47
ofrzeta wrote: post #6

Earlier quoted context omitted.

I've come to use more and more Python for tasks I would have used Shell in the past. Because of weird syntax with anything but simple variables or endless fiddling with quoting. You can get a lot done quite easily and concise in Python 3, for instance with the pathlib module that has methods for recursive globbing and stuff like that. https://docs.python.org/3.7/library/pathlib.html

Python needs to be installed, maintained, and every single library will have its list of dependencies (along with breaking API changes once in a while). There's a reason Bash exists: it's everywhere by default and you know it will still work as it works now in x years.

That's great, as long as you're happy with the POSIX Unix utilities (probably, wouldn't bet my life on it) and whatever Bash 3.x provides.

This can break quite fast if you rely on something from Bash 4 (released 11 years ago, still not on OS X by default), or some command-line tools that's either not installed or uses switches/functions only available in their GNU incarnations.

For anything that involves more than one file or with a line count going into triple digits, I wouldn't recommend bash. And heck, for those kind of scripts, I'd go one step further and stick with plain 'sh', just to be really sure. Bash adds little. GNU missed the train of transitioning to something like ksh93, probably because RMS thought that everyone would be using Guile once Hurd is out. (I once worked with a pretty large piece of software for packaking/administration mostly written in ksh93, as this was more palatable for commercial Unix users -- not as important these days where you won't run into AIX, HP-UX and Irix anymore)

For anything more involved than a few straightfoward lines, I'd probably still pick Perl 5. Very common, especially as some operating systems/distros have it in their core tooling (e.g. CentOS) and because it's included in git, so even available on Windows. Standard functionality is good enough and it's no big jump from sh/awk/sed. Rather do that before I depend on some obscure gawk function or gsed switch, especially as it's really hard to tell whether that's proprietary or in what version it was first introduced. Cross-browser checking is easier these days than cross-distro/unix.

Python does have a better/bigger standard library, but isn't as common and you might even run into Python 2/3 shenanigans these days.

Still, the situations where you have something moderately complex and can't install stuff should be pretty rare. Might not be able to do it for the whole system, but things like perlbrew/rbenv/nvm/pyenv-du-jour solve that mostly. At that level, Go might be an option, too.

Re: The Bash Hackers Wiki

#70

Post by ofrzeta »

hannob wrote: post #67
ofrzeta wrote: post #6

Earlier quoted context omitted.

I've come to use more and more Python for tasks I would have used Shell in the past. Because of weird syntax with anything but simple variables or endless fiddling with quoting. You can get a lot done quite easily and concise in Python 3, for instance with the pathlib module that has methods for recursive globbing and stuff like that. https://docs.python.org/3.7/library/pathlib.html

Actually I have recently analyzed some python code that does some shell calls and if I can eliminate them with pure python. It was quite frustrating. A lot of stuff that the shell has simple functionality for can't be done easily in python. Examples: * Call to chown -R. Sure, python can change file owners (os.chown()). But it has no parameter to do this recursively. You need to build a loop around it over some file i…

Fair enough, choosing the right tool for the job. On my own servers I can take care that the requires modules are installed and even use virtual envs for stable library versioning. This is how we do Let's Encrypt renewal because I can use a Python library for Route53 authentication that works.

Your first example could be solved with something like

[os.chown(p, uid, gid) for p in pathlib.Path('.').rglob('*')]

Obviously this is more verbose than chown but I think it's ok in a script.

Post reply on HN