Overhead of Python asyncio tasks
textual.textualize.io
Overhead of Python asyncio tasks
1–10 of 88 posts
Re: Overhead of Python asyncio tasks
#2> Clearly create_task is as close as you get to free in the Python world, and I would need to look elsewhere for optimizations. Turns out Textual spends far more time processing CSS rules than creating tasks (obvious in retrospect).
Takeaways:
1. Creating async tasks is cheap. 2. It is important to confirm intuitions, before acting on them.
Re: Overhead of Python asyncio tasks
#3I haven't used asyncio that much, certainly not in any serious sense, but wouldn't ContextVar lookups be a major factor of performance in serious asyncio code? Using tasks for things that aren't io-bound seems likely to give a false sense of performance superiority when doing basically nothing.
Re: Overhead of Python asyncio tasks
#4After running that code on both a Windows SB3 and major souped up Lenovo running Ubuntu...I just feel inadequate.
Re: Overhead of Python asyncio tasks
#5After running that code on both a Windows SB3 and major souped up Lenovo running Ubuntu...I just feel inadequate.
Windows SB3:
100,000 tasks 177,778 tasks per/s
200,000 tasks 150,588 tasks per/s
300,000 tasks 152,381 tasks per/s
400,000 tasks 134,031 tasks per/s
500,000 tasks 160,804 tasks per/s
600,000 tasks 129,293 tasks per/sRe: Overhead of Python asyncio tasks
#6After running that code on both a Windows SB3 and major souped up Lenovo running Ubuntu...I just feel inadequate.
Ubuntu:
100,000 tasks 155,257 tasks per/s
200,000 tasks 138,569 tasks per/s
300,000 tasks 134,779 tasks per/s
400,000 tasks 144,371 tasks per/s
500,000 tasks 135,672 tasks per/s
600,000 tasks 135,299 tasks per/s
700,000 tasks 146,456 tasks per/s
800,000 tasks 139,192 tasks per/sRe: Overhead of Python asyncio tasks
#7I haven't used asyncio that much, certainly not in any serious sense, but wouldn't ContextVar lookups be a major factor of performance in serious asyncio code? Using tasks for things that aren't io-bound seems likely to give a false sense of performance superiority when doing basically nothing.
I’d be surprised if context vars are more expensive than a dict lookup or two. But I haven’t profiled. Could be wrong.
Re: Overhead of Python asyncio tasks
#8async tasks are cool, but the usual PSA applies here:
Be careful to hold your references, because async tasks without active references will be garbage collected. I've been bitten by that in the past.
Long discussion here: https://bugs.python.org/issue21163
Docs: https://docs.python.org/3/library/asyncio-task.html#asyncio....
"Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done."
Re: Overhead of Python asyncio tasks
#9Mostly a testament to how absurdly fast modern CPUs are despite Python itself being so slow.
/ Recent Python convert, in spite of the horrible general performance of the official implementation of the language. That sweet, sweet module library. Also, with Docker containers the deployment issues have been solved. It might be slow to execute but it's really efficient to develop with.
Re: Overhead of Python asyncio tasks
#10Reading this is like reading early Renaissance alchemist arguing about how much mercury they need to combine with how much silver to create gold... This is so far gone I don't even know where to begin...
> It may be IO that gives AsyncIO its name, but Textual doesn't do any IO of its own.
So why on Earth are you using AsyncIO? You don't need it, if that's true...
> Those tasks are used to power message queues
How are your message queues not doing I/O? What on Earth are they doing then?
Needless to say that the whole benchmark is worthless because it never even initiates anything that would be involved when creating actual asynchronous I/O tasks...
----
I mean, I know, in Pythonland this is just your average Wednesday, but dear lord, if you don't visit that land all that often it shocks you more every time you do.