Show HN: Using Rust to write shell-script like tasks
1–10 of 83 posts
Re: Show HN: Using Rust to write shell-script like tasks
#2I love this. I tried to do something similar in Go, because it was in use on my team at Airbnb, and we were looking to port a 2000 line make-and-bash tool to... something not make-and-bash. But as you know, Go doesn’t have macros - so I spent all this effort trying to build a @decorator comment macro system in my personal time (abandoned). Rust seems like a perfect fit for this! We did have a teammate pitching rust, but no one wanted to learn it.
Anyways, congrats on the release. It looks fabulous. Safely splicing shell command snippets together is surprisingly annoying, so it’s really cool to see a hygienic yet user-friendly approach.
Re: Show HN: Using Rust to write shell-script like tasks
#3My god, this super useful when you have a mix o shell commands and processing text output from them. Bash isn’t particularly easy to work with parsing non-trivial strings in a readable way (I’m looking at you awk)
Re: Show HN: Using Rust to write shell-script like tasks
#4Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t.
It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-sh files stand out and bring attention to themselves as files that need to be audited. This wouldn’t. If you need to embed other languages or even just the approximate concept of then, then please at least keep those language files separate rather than inlining them.
Then you have an issue that people who are already aware of the pitfalls of shell scripts would know to read through any such scripts but this introduces a newer and unfamiliar scripting language to audit (eg how do we knew that what’s been declared is run but free?). At least Bash et al has had many years of eyeballs on it.
Re: Show HN: Using Rust to write shell-script like tasks
#5I am not a huge fan of copying the shell language wholesale and wrapping inside a macro. Since macros can execute arbitrary code, this makes me feel uneasy that the strings are just executed within a shell context, with all the appropriate, bug-prone, expansion done by the shell.
Seeing "ls /nofile || true;" makes me worry that "||" is actually passed to the shell wholesale. There's also no transparency about how the binary names are resolved.
I much prefer an approach more integrated with the language, like Plumbum: https://plumbum.readthedocs.io/en/latest/local_commands.html...
This no longer looks like the POSIX shell, but instead clearly integrates the good parts directly into the language, even if some complexity bubbles through. I don't have to worry that "grep["world"] < sys.stdin" is piped into an actual shell, because it gets converted into an AST on the way to execution.
Re: Show HN: Using Rust to write shell-script like tasks
#6Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t. It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-s…
> unsafe Rust ... since you’re now forking external code
Are you saying that Rust becomes unsafe because it used a C program as a subroutine? E.g. "tar xvf -" or whatever? What is the fix: rewrite tar, awk, scp and whatever else as Rust functions? That's a lot of work.
I'm surprised that you're simultaneously overlooking what ought to be a more gaping problem: that every system call made by a Rust program is a trip through a kernel written in C.
Re: Show HN: Using Rust to write shell-script like tasks
#7Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t. It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-s…
[deleted]
Re: Show HN: Using Rust to write shell-script like tasks
#8Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t. It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-s…
> unsafe Rust ... since you’re now forking external code Are you saying that Rust becomes unsafe because it used a C program as a subroutine? E.g. "tar xvf -" or whatever? What is the fix: rewrite tar, awk, scp and whatever else as Rust functions? That's a lot of work. I'm surprised that you're simultaneously overlooking what ought to be a more gaping problem: that every system call made by a Rust program is a trip t…
There’s a few problems with forking out:
1. Do those programs exist and what happened if they don’t? That behaviour is already understood in Bash, less so in random 3rd party Rust libraries.
2. Is ‘tar’ calling ./tar, /bin/tar or some other instance of tar? And how do you find out? (eg easy to check $PATH in Bash but does this library honour that? Easy to ‘which tar’ but is that going to be the same tar that this library forks?)
3. Are the people using this software even aware what external programs are being executed? How do they validate this? A .sh file clearly signals that there are external dependencies that need to be audited. A .rs file does not. This problem becomes magnified if you then start shipping compiled binaries rather than source.
I get people who like Rust are unlikely to be people who like writing shell scripts but the better way to think of this is like an MVC-like design where you have separate concerns that should be clearly separated in source.
Re: Show HN: Using Rust to write shell-script like tasks
#9Earlier quoted context omitted.
> unsafe Rust ... since you’re now forking external code Are you saying that Rust becomes unsafe because it used a C program as a subroutine? E.g. "tar xvf -" or whatever? What is the fix: rewrite tar, awk, scp and whatever else as Rust functions? That's a lot of work. I'm surprised that you're simultaneously overlooking what ought to be a more gaping problem: that every system call made by a Rust program is a trip t…
There’s a few problems with forking out: 1. Do those programs exist and what happened if they don’t? That behaviour is already understood in Bash, less so in random 3rd party Rust libraries. 2. Is ‘tar’ calling ./tar, /bin/tar or some other instance of tar? And how do you find out? (eg easy to check $PATH in Bash but does this library honour that? Easy to ‘which tar’ but is that going to be the same tar that this lib…
None of these are, as far as I can tell, "unsafe" in the rust sense. They won't result in memory safety issues in the current process.
Your concern is a generic concern about shelling out which maybe makes sense in some cases, but is untenable in general.
I also don't see how 1 & 2 are real problems. From looking at the readme I understand what happens if tar doesnt exist. The macro raises an error. This is similar to what calling subprocess.check_call would do in python. It's quite safe and well understood.
And 2 feels truly made up. Not only could you invoke "which tar" within the macro to find the answer, Id bet you a good bit of money that the answer is whatever is in your path. Anything else would be weirdly complicated. This is exactly the same as every other language that has a way to shell out to a subprocess.
And even if you have MVC like design where things are clearly separate, something will need to call the shell at some point. So issues 1, 2 and 3 never go away, even if you stick the script in a .sh file, you still need to invoke that file. And now how do you deploy that?
Re: Show HN: Using Rust to write shell-script like tasks
#10I am not a huge fan of copying the shell language wholesale and wrapping inside a macro. Since macros can execute arbitrary code, this makes me feel uneasy that the strings are just executed within a shell context, with all the appropriate, bug-prone, expansion done by the shell. Seeing "ls /nofile || true;" makes me worry that "||" is actually passed to the shell wholesale. There's also no transparency about how the…
I agree, this is a misguided idea. The whole point of not using Bash is that you don't have to use it's terrible design and syntax surely?