How to Make Python Wait

Good article on an important topic for novices, but why is it so long ?

The general strategy of "use the OS threading primitives" like join and events works in most languages.

Actually, I started reading about async/await from asyncio package. But for a newbie the above article is much simpler to grasp concurrency and parallelism concepts. And the sample code makes it a lot easier.

Don't follow this blog post advice.

Manually dealing with threads and processes is useful if you want to build a framework or a very complex workflow. But chances are you just want to run stuff concurrently, in the background.

In that case (which is most people case), you really want to use one of the stdlib pools: it takes care of sync, serialization, communication with queues, worker life cycle, task distribution, etc. for you.

Plus in such case, waiting for results is super easy:

    import time
    import random

    from concurrent.futures import ThreadPoolExecutor, as_completed

    # the work to distribute
    def hello():
        seconds = random.randint(0, 5)
        print(f'Hi {seconds}s')
        time.sleep(seconds)
        print(f'Bye {seconds}s')
        return seconds

    # max concurrency is 2
    executor = ThreadPoolExecutor(max_workers=2)
   
    # submit the work
    a = executor.submit(hello)
    b = executor.submit(hello)

    # and here we wait for results
    for future in as_completed((a, b)):
        print(future.result())

Want multiple process instead ? It's the same API:

    import time
    import random

    from concurrent.futures import ProcessPoolExecutor, as_completed

    def hello():
        seconds = random.randint(0, 5)
        print(f'Hi {seconds}s')
        time.sleep(seconds)
        print(f'Bye {seconds}s')
        return seconds

    # Don't forget this for processes, or you'll get in trouble
    if __name__ == "__main__":

        executor = ProcessPoolExecutor(max_workers=2)

        a = executor.submit(hello)
        b = executor.submit(hello)

        for future in as_completed((a, b)):
            print(future.result())

This is Python. Don't make your life harder that it needs to be.

This is solid advice.