Python Asyncio

You wouldn't do it that way for any reason other than you can get better performance than way than by using threading or other pre-emptive multitasking.

It's not a better developer experience than writing straight forward code where threads are linear and interruptions happen transparently to the flow of the code. There are foot guns everywhere, with long running tasks that don't yield or places where there are hidden blocking actions.

It reminds me a bit of the old system 6 mac cooperative multitasking. It was fine, and significantly faster because your program would only yield when you let it do it, so critical sections coule be guaranteed to not context shift. However, you could bring the entire machine to a halt by holding down the mouse button, as eventually an event handler would get stuck waiting for mouse up.

Pre-emptive multitasking was a huge step forward -- it made things a bit slower on average, but the tail latency was greatly improved, because all the processes were guaranteed at least some slice of the machine.

I'd need to know the methodology. Asyncio is "gated", meaning that coroutines execute in tranches. When you queue up a bunch of coroutines from another coroutine, they don't execute right away instead they go in a list. Then when the current tranche completes the next one starts. There's some tidying up which occurs with every tranche.

As an (only one) example of "measuring the wrong thing", if you queue up a single task 1000 times which takes as its argument a unique timer instance it matters whether you queue them up one at a time as they finish running or queue them all up at once.

    #!/usr/bin/python3
    # (c) 2022 Fred Morris, Tacoma WA USA. Apache 2.0 license.
    """Illustrating the tranche effect in asyncio."""

    from time import time
    import asyncio

    N = 1000

    class TimerInstance(object):
        def __init__(self, accumulator):
            self.accumulator = accumulator
            self.start = time()
        def stop(self):
            self.accumulator.cumulative += time() - self.start
            return
        
    class Timer(object):
        def __init__(self):
            self.cumulative = 0.0
            return
        def timer(self):
            return TimerInstance(self)

    async def a_task(timer):
        timer.stop()
        return

    def main():
        loop = asyncio.get_event_loop()
        timing = Timer()
        overall = time()
        for i in range(N):
            loop.run_until_complete( loop.create_task( a_task(timing.timer()) ) )
        print('Sequential: {}   Overall: {}'.format(timing.cumulative, time() - overall))
        timing = Timer()
        overall = time()
        for i in range(N):
            loop.create_task( a_task(timing.timer()) )
        loop.stop()
        loop.run_forever()
        print('Tranche: {}   Overall: {}'.format(timing.cumulative, time() - overall))

    if __name__ == '__main__':
        main()


    # ./tranche-demo.py 
    Sequential: 0.02229022979736328   Overall: 0.04146838188171387
    Tranche: 6.084041595458984   Overall: 0.012317180633544922

The docs on that page also point you to a different task library which does what I'd expect:

"If the work is appropriate for concurrency and parallelism, also consider using the Task Parallel Library."

I checked those docs and that is a framework that makes sense.

Reluctant to add a "me-too" comment but I think you have it there. For whatever reason people are much keener to investigate different ways of scheduling IO than they are to investigate what the query plan looks like and why.