> Problem 3: Maintaining two APIs is hard
Well I had a fix https://news.ycombinator.com/item?id=43982570
Python has had async for 10 years – why isn't it more popular?
221–230 of 305 posts
Re: Python has had async for 10 years – why isn't it more popular?
#222The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…
also most python usecases that are in the realm of things like high performance concurrent request servicing push it down into libraries that i think are often tied to a native network request processing core. (gunicorn, grpc, etc)
python is kind of a slow choice for that sort of thing regardless and i don't think the complexity of async is all that justified for most usecases.
i still maintain my position that a good computer system should let you write logic synchronously and the system will figure out how to do things concurrently with high performance. (although getting this right would be very hard!)
Re: Python has had async for 10 years – why isn't it more popular?
#223[deleted]
Re: Python has had async for 10 years – why isn't it more popular?
#224The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…
It might have been too little but it wasn’t too late.
Generations of programmers have given up on downloading data async in their Python scripts and just gone to bash and added a & at the end of a curl call inside a loop.
Re: Python has had async for 10 years – why isn't it more popular?
#225I don't love Python's async API either, but I think a lot of its complained-about complexity arises from two things: making "when does the coroutine start running" a very explicit point in code (hence the Task/awaitable-function dichotomy), and how it chooses to handle async cancellation: via exceptions.
And Python's async cancellation model is pretty nice! You can reason about interruptions, timeouts, and the like pretty well. It's not all roses: things can ignore/defer cancellations, and the various wrappers people layer on make it hard to tell where, exactly, Tasks get cancelled--awaitable functions are simple here, at least. But even given that, Python's approach is a decent happy medium between Node's dangling coroutines and Rust's no-cleanup-ever disappearing ones (glib descriptor: "it's pre-emptive parallelism, but without the parallelism").
More than a little, I think, of the "nobody does it this way" weirdness and frustration in Python asyncio arises from that. That doesn't excuse the annoyances imposed by the resulting APIs, but it is good to know.
Re: Python has had async for 10 years – why isn't it more popular?
#226Earlier quoted context omitted.
async is like a virus. I think the implementation in js and .NET is somewhat ok’ish because your code is inside an async context most of the time. I really hate the red / blue method issues where library functions get harder to compose. Oh I have a normal method because there was no need for async. Now I change the implementation and need to call an async method. There are ways around this but more often than not wil…
It is not nearly as much of a problem in JS because JS only has an event loop, there is no way to mix in threads with async code because there are no threads. Makes everything a lot simpler and a lot of the data structures a lot faster (because no locks required). But actual parallelization (instead of just concurrency) is impossible[1]. A lot of the async problems in other languages is because they haven't bought up…
In my less charitable moments, I've wondered if the real reason Python has async/await is because people coming to it from JavaScript couldn't be arsed to learn a more appropriate paradigm.
Re: Python has had async for 10 years – why isn't it more popular?
#227Not too long ago, I read a comment on HN that suggested, due to Python's support for free-threading, async in Python will no longer be needed and will lose out to free-threading due to it's use of "colored" functions. Which seems to align with where this author ends up: > Because parallelism in Python using threads has always been so limited, the APIs in the standard library are quite rudimentary. I think there is an…
C# has had free threading all along, yet still saw the need for async as a separate facility. The same goes for C++, which now has co_await.
Threads are more expensive and slow to create. Submitting a task to a thread pool and waiting for a result, or a bunch of results, to show up, is much more ergonomic. So `async` automatically submits a task, and `await` awaits until it completes. Ideally `await` just discovers that a task (promise) has completed at that point, while the main thread was doing other things.
Once you have this in place, you can notice that you can "submit the task to the same thread", and just switch between tasks at every `await` point; you get coroutines. This is how generators work: `yield` is the `await` point.
If all the task is doing is waiting for I/O, and your runtime is smart enough to yield to another coroutine while the I/O is underway, you can do something useful, or at least issue another I/O task, not waiting for the first one to complete. This allows typical server code that does a lot of different I/O requests to run faster.
Older things like `gevent` just automatically added yield / await points at certain I/O calls, with an event loop running implicitly.
Re: Python has had async for 10 years – why isn't it more popular?
#228> If you call function get_thing_sync() versus await get_thing_async(), they take the same amount of time.
No, if you call both function one will try and fetch a none responding url and the other will immediately raise an exception.
Re: Python has had async for 10 years – why isn't it more popular?
#229I don't love Python's async API either, but I think a lot of its complained-about complexity arises from two things: making "when does the coroutine start running" a very explicit point in code (hence the Task/awaitable-function dichotomy), and how it chooses to handle async cancellation: via exceptions. And Python's async cancellation model is pretty nice! You can reason about interruptions, timeouts, and the like p…
Cancellations probably caused more bugs in my async code than anything else.
If any code in your coroutine, including library code, has a broad try/except, there's good chances that eventually the cancellation exception will be swallowed up and ignored.
Catch-all try/except of course isn't the pinnacle of good software engineering, but it happens a lot, in particular in server-tyoe applications. You may have some kind of handler loop that handles events periodically, and if one such handling fails, with an unknowabl exception, you want to log it and continue. So then you have to remember to explicitly reraise cancellation errors.
Maybe it's the least bad Pythonic option, but it's quite clunky for sure.
Re: Python has had async for 10 years – why isn't it more popular?
#230Because it's a niche? You don't need async for most stuff Python is used for, it's a "nice-to-have", and it's annoying to add. If you have to have concurrency/threading/etc, there are other languages with better paradigms. The same thing happened with Perl and its weird threading (for different reasons, but still)... I guess Python didn't learn that lesson. Perl also gained async and coroutine support, but I think th…
Async is perfectly fine for a medium-load single-threaded RPC/REST server wrangling. Ran it in production with 100s, sometimes 1000s of calls / sec with no issues. And thread-safety is much easier with async, where you know where the context switches are occuring.