I would prefer to implement it using a synchronous approach and then switch to asynchronous at deployment.
Python has had async for 10 years – why isn't it more popular?
241–250 of 305 posts
Re: Python has had async for 10 years – why isn't it more popular?
#242Earlier quoted context omitted.
> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…
I'm a person who wrote an entire networking library in Python and I agree with you. The most obvious issue with Python's single-threaded async code is any slow part of the program delays the entire thing. And yeah -- that's actually insanely frigging difficult to avoid. You write standard networking code and then find out that parts you expected to be async in Python actually ended up being sync / blocking. DESPITE T…
bottle.py is a WSGI backed framework, right? So it's agnostic about whether you are running with threads, fork, blocking single thread IO, gevent, or what.
Re: Python has had async for 10 years – why isn't it more popular?
#243Earlier quoted context omitted.
I'm a person who wrote an entire networking library in Python and I agree with you. The most obvious issue with Python's single-threaded async code is any slow part of the program delays the entire thing. And yeah -- that's actually insanely frigging difficult to avoid. You write standard networking code and then find out that parts you expected to be async in Python actually ended up being sync / blocking. DESPITE T…
bottle.py is a WSGI backed framework, right? So it's agnostic about whether you are running with threads, fork, blocking single thread IO, gevent, or what.
Umm, acktually... (the default server is non-threaded and sequential. It was an example.)
Re: Python has had async for 10 years – why isn't it more popular?
#244The main reason I've encountered is that async is generally an all-or-nothing thing. It's generally not possible to take an existing code base and just "add a little async here and there". The entire thing has to be restructured from the ground up for async. The gain has to be really major for this to be worthwhile, more so the better your existing code already works.
This is sort of like the article's Problem 3, but it's not just maintaining two APIs, it's even creating the second API in the first place.
Re: Python has had async for 10 years – why isn't it more popular?
#245It was supposed to bring massive concurrency to Python. But as with any async implantation in any language it is too easy to deadlock the entire system. Did you forgot to sprinkle enough `await`? Your code is blocked somewhere, good luck hunting for it. In contrast preemptive green threads are too easy. Be it IO or CPU load all threads will get their slice of CPU time. Nothing is blocked so you can debug your logic e…
What I find funny is Java started with green threads, then moved away to system threads, then back again.
Maybe massive concurrency was not that big of a feature back then. But these days everyone wants to support a million connections at a time. Green threads and async tasks can do that without breaking a sweat, unlike OS threads. Also, Java virtual threads are still cooperative. Maybe they will move to preemption in time like Go did.
Some time ago I tried to run just 10k OS threads on a small PC and it just crashed. So clearly OS threads have not improved much.
Re: Python has had async for 10 years – why isn't it more popular?
#246Earlier quoted context omitted.
GIL is not part of the language design, it's just a detail of the most common implementation - CPython.
Fair and accurate. But that’s pretty much what people use, right? I am happy to hear stories of using pypy or something to radically improve an architecture. I don’t have any from personal experience. I guess twisted and stackless, a long time ago.
The GIL is optional in new Python versions. Downsides are legacy library compatibility and degraded single thread performance.
Re: Python has had async for 10 years – why isn't it more popular?
#247Python async feels great ergonomically at first. Async tasks, timers, things happening concurrently. But then you lose stack traces. Code that starts sync, then appends jobs to an async event queue can't be traced, because the exception happened in the loop and doesn't bubble up to sync (or it didn't back in 3.8). That, among with synchronization problems (it is an event loop, why do we even need to synchronize? async mutex??)
I really tried to make it work, but eventually gave up and went back to deque. Now life is great.
Re: Python has had async for 10 years – why isn't it more popular?
#248I used to keep plugging Unyielding [1] vs. What Color Is Your Function [2] as the right matrix to view these issues within. But then Notes on structured concurrency [3] was written and I just point to that these days. But, to sum it all up for those who want to talk here, there are several ways to look at concurrency but only one that matters. Is my program correct? How long will it take to make my program correct? S…
Man, that Trio [3] read was great. When we demand that all concurrent tasks must join, then we can better reason about our programs.
I already kinda had this idea while working with Rust. In Rust, Futures won’t execute unless `await`ed. In practice, that meant that all my futures were joined. It was just the only way I could wrap my head around doing anything useful with async.
Re: Python has had async for 10 years – why isn't it more popular?
#249Earlier quoted context omitted.
No it was a js dev complaining about callbacks in node. Mainly because a lot of standard library code back then only came in callback flavour. i.e. no sync file writes, etc.
Which is really funny because the linux kernel doesn't do async for file writes :D
Uh, io_uring does that just fine.