21–30 of 47 posts
Having the workers acquire the lock and update the terminal themselves seems like it would cause lock contention. An alternative would be to have only the main process do the updating and have the workers message it about progress, using a queue.
You probably don't want to run this gist directly. Looks like a risk of a runaway process creation, depending on the platform. The process creation code should be guarded by `if __name__ == "__main__"`.
I've got the impression that in practice, what keeps developers from letting trivially parallelizable tasks run in parallel is a) the overhead of dealing with poor parallelization primitives and b) the difficulty in properly showing the status of parallel invocations. Having good features to support this in standard libraries would go a long way to incentivizing devs to actually parallelize.
cool call me when they got one for go lol
I've got the impression that in practice, what keeps developers from letting trivially parallelizable tasks run in parallel is a) the overhead of dealing with poor parallelization primitives and b) the difficulty in properly showing the status of parallel invocations. Having good features to support this in standard libraries would go a long way to incentivizing devs to actually parallelize.
I've got the impression that in practice, what keeps developers from letting trivially parallelizable tasks run in parallel is a) the overhead of dealing with poor parallelization primitives and b) the difficulty in properly showing the status of parallel invocations. Having good features to support this in standard libraries would go a long way to incentivizing devs to actually parallelize.
We do see some cool stuff under the hood from core Python devs but interest in further quality of life features seems to be lacking.
I've got the impression that in practice, what keeps developers from letting trivially parallelizable tasks run in parallel is a) the overhead of dealing with poor parallelization primitives and b) the difficulty in properly showing the status of parallel invocations. Having good features to support this in standard libraries would go a long way to incentivizing devs to actually parallelize.
For a one off project it seems simpler to just write an html UI.
Having the workers acquire the lock and update the terminal themselves seems like it would cause lock contention. An alternative would be to have only the main process do the updating and have the workers message it about progress, using a queue.
You could also just send output to separate files by re-opening stdout/stderr and use https://www.vanheusden.com/multitail/index.html (or GNU screen or tmux or whatever) to multiplex a terminal in a more organized way. This also solves an "open problem" in the article of stray prints. If you really want to share one terminal / stdout but also prevent timing-based record splitting, you could also send outputs to FIFOs…