Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
1–10 of 79 posts
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#2Just to note the it's the same in Rust with tokio::spawn (and alternatives).
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#3The article’s understanding of Java’s Virtual Threads is incorrect. Virtual threads have a preemptive concurrency model, not a co-operative one, so suspension points are not only at calls to things like Future.get()
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#4Sorry if I've got this wrong, but wouldn't the first example behave the same way in Javascript as well? The function parent() "awaits" the completion of child(), so it wouldn't be possible to interleave the print statements.
The example from this StackOverflow question might be a better demonstration: https://stackoverflow.com/q/63455683
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#5This is gold.
Here’s a horror story for you.
A few years ago I worked for this startup as a principal engineer. The engineering manager kept touting how he was from XYZ and that he ran Pythonista meetups and how vast his knowledge of Python was. We were building a security product and needed to scan hundreds of thousands of documents quickly so we built a fan out with coroutines. I came on board well into this effort to assist in adding another adapter to this other platform that worked similarly. After seeing all the coroutines, being pickled and stored in S3, so that nodes could “resume” if they crashed yet - not a single create_task was present. All of this awaiting, pickling, attempting to resume, check, stuff, report, pickle, happened synchronously.
When trying to point out the issue with the architecture and getting into a shouting match with Mr. Ego, I was let go.
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#6For the JS developers: similar useful behavior (and more!) can be implemented in JS using the wonderful Effection library.
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#7Maybe I'm nitpicking a bit but the second concrete example shows different flow in the parent method compared to the first one. If the second example would be a modification of the first one like shown below the output would be the same in both cases.
async def parent():
print("parent before")
task = asyncio.create_task(child()) # Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#8Personally, I've never been able to make async work properly with Python. In Node.js I can schedule enough S3 ListBucket network requests in parallel to use 100% of my CPU core, by just mapping an array of prefixes into an array of ListBucket Promises. I can then do a Promise.all() and let them happen.
In Python there's asyncio vs threading, and I feel there's just too much to navigate to quickly get up and running. Do people just somehow learn this just when they need it? Is anyone having fun here?
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#9I may be misunderstanding it, but the examples shown don't seem to be illustrative? It seems reasonably obvious that the prints will happen in this order in any language, because in example 1 we are explicitly (a)waiting for the child to finish, and in example 2 both of the parent prints are above the await. So I don't feel like either of these makes the point the author is trying to get across?
Re: Await Is Not a Context Switch: Understanding Python's Coroutines vs. Tasks
#10but how would you systematically audit a large async codebase to find all the hidden interleave points, unnecessary locks?