Live data from Hacker News

Why Create a New Unix Shell?

oilshell.org

251–260 of 298 posts

Re: Why Create a New Unix Shell?

#251
post #212
post #136

@chubot: There is one use-case I come across every once in a while: https://stackoverflow.com/questions/356100/how-to-wait-in-ba... So whenever you want to do things in parallel there is probably a limit to the number of processes you would like to execute in parallel (e.g. the famous compiler limit formula: number of CPU cores +1). It would be great if Oil could support such a use-case out of the box, as easy parall…

Absolutely. In fact, the bash manual explicitly refers to GNU parallel for this use case! I use xargs -P all over the Oil codebase, which does what you want. The trick is to end the file with "$@", and then invoke xargs -P 4 -- $0 my-func. That way xargs can run arbitrary shell functions, not just something like sh -c "..." ! I'm going to write a blog post about this. I also do this with find -exec $0 myfunc ';' http…

Sounds pretty cool. My biggest problems with xargs is that I had constantly some weird edge cases, so I try to avoid it. As GNU parallel doesn't seem to be part of standard installations it would be an external dependency for a script, which I try to avoid too.

So I ended up using the loop syntax:

  for i in {0..9}; do
    echo "$i" &
  done
  wait
It is not so Unix like, but I find it easier to debug. It would be great if Oil would have a solution for limiting that kind of parallel execution too. I am aware that this isn't simple as there are different options here how to implement it (global limit vs. local limit vs. named limit).

Just an idea from the top of my head an idea for an optional named limit:

Lets call it 'flow':

  flow [options] [command]':
  -n number of max parallel processes
  -c (optional) identifier of the counter.

  Example:

  for i in {0..9}; do
    flow -c myCounter -n 4 echo "$i"
  done
Just an idea.

Re: Why Create a New Unix Shell?

#252

Earlier quoted context omitted.

> an IDE based on the language it was written in I recommend Emacs. Maybe not as a daily driver (it's a matter of preference), but for the experience. I recommend at least a month with it, make sure to write some original Lisp code for your customisations. (You WILL end up customizing it, the defaults are crap.) > or a window manager I recommend Awesome. The core is in C, but that's basically the low-level stuff, the…

> the defaults are crap As someone who has recently joined the 1k LOC Emacs configuration file club, I have to disagree. The default settings are a well thought out starting point that needs minimal tweaking to get to exactly where you want. I try out a lot of packages on ELPA, and many that purport to provide alternatives for defaults end up being inferior to using the defaults with some small tweaks. The convenient…

My favourite Emacs goodie to highlight is magit because it really changed the way I use git. All of a sudden, it's incredibly simple to (un-)stage individual hunks, etc. And of course it works over TRAMP, too :)

org-mode is a good one, too, but I feel like I've barely scratched the surface there.

Re: Why Create a New Unix Shell?

#253

Earlier quoted context omitted.

yeah I can actually imagine that. like using an IDE based on the language it was written in to extend and tweak it more easily, or a window manager etc.. actually now you mentioned it, barring possible performance issues I'd also like a Python distro

Early Smalltalk and Lisp systems were those langs all the way down. Using emacs with Common Lisp or using Pharo Smalltalk can give you a feel for this, but not exactly the same thing as a Symbolics lisp machine or the Xerox Alto. Alan Kay has a presentation where he shows the entire OS, word processor, networking stuff, IDE, paint programs...etc was an insanely small amount of code and everything was user configurabl…

I feel you. This guy is working on an interesting project called the Reform laptop [1]. You might have seen this already. He will eventually have it run his own lisp (called "Interim") all the way down.

[1] http://mntmn.com/reform/

Re: Why Create a New Unix Shell?

#254

Lots of overlap in design goals with fish, except fish also places a premium on users interactively using the shell (which means friendlier in-repl experience but a balancing act when it comes to features). Fish’ auto completions are incredible, too. Best of luck to them. Another interesting shell to check out is elvish, lots of new ideas there (even if awkward to use). (Disclosure: I’m one of the core fish devs/main…

Thanks for your work on fish. I've been using it for some time and am very happy with it.

What's the process for requesting functions be added to core? I had to write my own to get bash's 'dirs -v' functionality. My solution depends on sed and is no doubt a hack.

Re: Why Create a New Unix Shell?

#255

Earlier quoted context omitted.

Back when I started programming, I wrote 1000+ lines of shell scripts. Now, I quite seriously believe that a 1000 line shell script only exists out of error. I still occasionally end up doing 2-300 line dense shell scripts, but not without feeling very dirty along the way. Either split into small, simple shell scripts (which is fine), or a different language. In the cross platform build pipeline at work, I keep a str…

> I quite seriously believe that a 1000 line shell script only exists out of error. Perhaps the best use case for Oil is to provide a debugging environment where you can figure out what your legacy shell scripts are doing, and rewrite them in another language. > Unlearning bash-isms is not a liberty I can afford, as I need to be proficient when I SSH into a machine I do not own or control. This is a slippery slope. I…

> "don't make your own custom aliases / shell functions, because they won't be available when you SSH to another machine."

If you're logging into a machine to the point you're editing files or require your customizations in this day and age you're doing it wrong IMHO.

Re: Why Create a New Unix Shell?

#256

Earlier quoted context omitted.

Back when I started programming, I wrote 1000+ lines of shell scripts. Now, I quite seriously believe that a 1000 line shell script only exists out of error. I still occasionally end up doing 2-300 line dense shell scripts, but not without feeling very dirty along the way. Either split into small, simple shell scripts (which is fine), or a different language. In the cross platform build pipeline at work, I keep a str…

Bash is flat-out not a scripting language . It is a command language . It does not support typed variables, or named parameters, or any number of basic scripting language features. It technically does not even provide an 'if' construct. It's sufficient as a 'glue layer', and the Unix toolchain is nice, but it provides next to nothing in the way of abstraction, and that's liable to become a problem closer to the 1000…

Well, Perl back in the bad old days (4 and earlier) was very similar in terms of limited "minimum standard" facilities, but tons of people used it as a general purpose scripting language and hacked together some really huge systems.

It's definitely easier/safer to write longer programs in other languages, but with proper discipline it's also possible to write really big, robust programs in Bash/shells--people have done this, and some of that code is probably still running today.

Re: Why Create a New Unix Shell?

#257

Earlier quoted context omitted.

Bash is flat-out not a scripting language . It is a command language . It does not support typed variables, or named parameters, or any number of basic scripting language features. It technically does not even provide an 'if' construct. It's sufficient as a 'glue layer', and the Unix toolchain is nice, but it provides next to nothing in the way of abstraction, and that's liable to become a problem closer to the 1000…

Well, Perl back in the bad old days (4 and earlier) was very similar in terms of limited "minimum standard" facilities, but tons of people used it as a general purpose scripting language and hacked together some really huge systems. It's definitely easier/safer to write longer programs in other languages, but with proper discipline it's also possible to write really big, robust programs in Bash/shells--people have do…

> and some of that code is probably still running today.

A lot of that code is still being written today, which is one of the things that OP is trying to change.

Re: Why Create a New Unix Shell?

#258

Earlier quoted context omitted.

Since python3.6 it's just subprocess.run(). The SP module was refactored. I hope that eliminates the need for all these annoying (IMHO) wrappers.

Ah, that's nice! ... unfortunately, I'm stuck with python2.4 or 2.6 on most machines my Python code has to run on. :/

> I'm stuck with python2.4 or 2.6

Not even 2.7?! 2.6 hasn't had a security update since 2013[0], I dread to think how old 2.4 is.

[0]: https://www.python.org/download/releases/2.6.9/

Re: Why Create a New Unix Shell?

#259

Earlier quoted context omitted.

Back when I started programming, I wrote 1000+ lines of shell scripts. Now, I quite seriously believe that a 1000 line shell script only exists out of error. I still occasionally end up doing 2-300 line dense shell scripts, but not without feeling very dirty along the way. Either split into small, simple shell scripts (which is fine), or a different language. In the cross platform build pipeline at work, I keep a str…

I agree that the age of 1000+ lines of shell script code should be over; just replace the shebang and use whatever other language you like instead. Unless you need something dependency-free, compilation-free, and portable, then God help you because sh it is. It's funny though, after decades of shying away from them I've now gone full circle and embraced Makefiles once more. Perhaps it's the cleanness of the bmake/pma…

> dependency-free, compilation-free, and portable, then God help you because sh it is.

That's a fair point. Nothing beats shells for ubiquity and supportedness everywhere . . . in theory.

In practice, even people who write "portable shell scripts" (almost) never really write portable shell scripts.

If you learn the POSIX sh standard like the back of your hand, and stick only to the features in it, never using bashisms or equivalent, most shellscripts still rely on external programs (even if only grep/sed/etc) to do their heavy lifting.

And that's where you get into trouble. Because compared to the variability in behavior of even "standard" ultra-common programs, the variability in behaviors between shell syntaxes/POSIX-vs-non-POSIX shells is tiny. The instant your code invokes an external program, no matter how ubiquitous that program is, you have to worry about:

- People screwing with PATH and changing the program you get.

- People screwing with variables that affect the external program's behavior: LD_LIBRARY_PATH for dynamic executables, or language-specific globals for programs' runtime behavior (CLASSPATH, PERL5LIB, PYTHONPATH, etc.).

- External program "editions" (e.g. non-GNU sed vs GNU sed). Good luck using PCRE with a non-GNU "grep"! Oh, and if you're sticking to POSIX-only shell semantics (no bashisms), you don't even get full BREs in most places you need them in the shell; you're stuck with limited BRE or globbing, which makes editing 100-line PCRE regexes feel like a dream.

- External program per-version differences.

- External program configuration.

- Etc.

Dealing with those issues is where "self-test"/"commandline program feature probe" things like autotools really shine. Raw shellscripts, though, very seldom live up to the "ubiquity" promise.

Re: Why Create a New Unix Shell?

#260

Earlier quoted context omitted.

Ah, that's nice! ... unfortunately, I'm stuck with python2.4 or 2.6 on most machines my Python code has to run on. :/

> I'm stuck with python2.4 or 2.6 Not even 2.7?! 2.6 hasn't had a security update since 2013[0], I dread to think how old 2.4 is. [0]: https://www.python.org/download/releases/2.6.9/

Python2.4 is what RHEL/CentOS 5 has to work with... And the extended support for that bloody release lasts till 2021.

You can install a newer version, but our CentOS 5 machines are meant for compatibility tests, so modifying their setup is unacceptable.

Post reply on HN