@chubot: There is one use-case I come across every once in a while: https://stackoverflow.com/questions/356100/how-to-wait-in-ba... So whenever you want to do things in parallel there is probably a limit to the number of processes you would like to execute in parallel (e.g. the famous compiler limit formula: number of CPU cores +1). It would be great if Oil could support such a use-case out of the box, as easy parall…
Absolutely. In fact, the bash manual explicitly refers to GNU parallel for this use case! I use xargs -P all over the Oil codebase, which does what you want. The trick is to end the file with "$@", and then invoke xargs -P 4 -- $0 my-func. That way xargs can run arbitrary shell functions, not just something like sh -c "..." ! I'm going to write a blog post about this. I also do this with find -exec $0 myfunc ';' http…
So I ended up using the loop syntax:
for i in {0..9}; do
echo "$i" &
done
wait
It is not so Unix like, but I find it easier to debug. It would be great if Oil would have a solution for limiting that kind of parallel execution too. I am aware that this isn't simple as there are different options here how to implement it (global limit vs. local limit vs. named limit).Just an idea from the top of my head an idea for an optional named limit:
Lets call it 'flow':
flow [options] [command]':
-n number of max parallel processes
-c (optional) identifier of the counter.
Example:
for i in {0..9}; do
flow -c myCounter -n 4 echo "$i"
done
Just an idea.