Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python has had async for 10 years – why isn't it more popular?

tonybaloney.github.io

Post reply on HN
Print view View original

61–70 of 305 posts

Re: Python has had async for 10 years – why isn't it more popular?

#61

Post by rsyring »

Not too long ago, I read a comment on HN that suggested, due to Python's support for free-threading, async in Python will no longer be needed and will lose out to free-threading due to it's use of "colored" functions. Which seems to align with where this author ends up:

> Because parallelism in Python using threads has always been so limited, the APIs in the standard library are quite rudimentary. I think there is an opportunity to have a task-parallelism API in the standard library once free-threading is stabilized.

> I think in 3.14 the sub-interpreter executor and free-threading features make more parallel and concurrency use cases practical and useful. For those, we don’t need async APIs and it alleviates much of the issues I highlighted in this post.

Armin recently put up a post that goes into those issue in more depth: https://lucumr.pocoo.org/2025/7/26/virtual-threads/

Which lead me to a pre-PEP discussion regarding the possibility of Virtual Threads in Python, which was probably way more than I needed to know but found interesting: https://discuss.python.org/t/add-virtual-threads-to-python/9...

Re: Python has had async for 10 years – why isn't it more popular?

#62

Post by kasperni »

JackSlateur wrote: post #42
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

green thread have pitfalls too, like this: https://news.ycombinator.com/item?id=39008026

This was a known issue and was fixed in Java 24 [1].

[1] https://openjdk.org/jeps/491

Re: Python has had async for 10 years – why isn't it more popular?

#64

Post by pnathan »

atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

Async taints code, and async/await fall prey to classic cooperative multitasking issues. "What do you mean that this blocked that?"

The memory and execution model for higher level work needs to not have async. Go is the canonical example of it done well from the user standpoint IMO.

Re: Python has had async for 10 years – why isn't it more popular?

#65

Post by nromiun »

ddorian43 wrote: post #33
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

Because it sucks compared to gevent (green threads). But for some reason, people always disregard this option. They don't even read it. Like any comment with gevent is shadowbanned and it doesn't register in their mind.

Gevent is too underrated. Even if people don't like the monkey patching you can simply use the gevent namespace API as well. No idea why people prefer the absolute mess that is Python async ecosystem.

Re: Python has had async for 10 years – why isn't it more popular?

#66

Post by throw-qqqqq »

lacker wrote: post #54
throw-qqqqq wrote: post #30

Earlier quoted context omitted.

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…

I'm confused, I feel like the two of you are expressing opposite opinions. The comment you are responding to prefers green threads to be managed like goroutines, where the code looks synchronous, but really it's cooperative multitasking managed by the runtime, to explicit async/await. But then you criticize "code that looks synchronous but is really async". So you prefer the explicit "async" keywords? What exactly is…

First, I don’t mean to criticize anything or anyone. People value such things subjectively, but for me the async/sync split does no good.

Goroutines feel like old-school, threaded code to me. I spawn a goroutine and interact with other “threads” through well defined IPC. I can’t tell if I’m spawning a green thread or a “real” system thread.

C#’s async/await is different IMO and I prefer the other model. I think the async-concept gets overused (at my workplace at least).

If you know Haskell, I would compare it to overuse of laziness, when strictness would likely use fewer resources and be much easier to reason about. I see many of the same problems/bugs with async/await..

Re: Python has had async for 10 years – why isn't it more popular?

#67

Post by [deleted] »

rybosome wrote: post #26

I suppose my negative experiences with async fall under #3, that it is hard to maintain two APIs. One of the most memorable "real software engineering" bugs of my career involved async Python. I was maintaining a FastAPI server which was consistently leaking file descriptors when making any outgoing HTTP requests due to failing to close the socket. This manifested in a few ways: once the server ran out of available f…

[deleted]

Re: Python has had async for 10 years – why isn't it more popular?

#68

Post by nly »

tdumitrescu wrote: post #22
PaulHoule wrote: post #4

I went through a phase of writing asyncio servers for my side projects. Probably the most fun I had was writing things that were responsive in complex ways, such as a websockets server that was also listening on message queues or on a TCP connection to a Denon HEOS music player. Eventually I wrote an "image sorter" that I found was hanging up when the browser was trying to download images in parallel, the image servi…

That's the main problem with evented servers in general isn't it? If any one of your workloads is cpu-intensive, it has the potential to block the serving of everything else on the same thread, so requests that should always be snappy can end up taking randomly long times in practice. Basically if you have any cpu-heavy work, it shouldn't go in that same server.

OS threads are for CPU bound work.

Async is for juggling lots of little initialisations, completions, and coordinating work.

Many apps are best single threaded with a thread pool to run (single threaded) long running tasks.

Re: Python has had async for 10 years – why isn't it more popular?

#69

Post by b33j0r »

atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

For me, once I wanted to scale asyncio within one process (scaling horizontally on top of that), only two things made sense: rust with tokio or node.js.

Doing async in python has the same fundamental design. You have an executer, a scheduler, and event-driven wakers on futures or promises. But you’re doing it in a fundamentally hand-cuffed environment.

You don’t get benefits like static compilation, real work-stealing, a large library ecosystem, or crazy performance boosts. Except in certain places in the stack.

Using fastapi with async is a game-changer. Writing a cli to download a bunch of stuff in parallel is great.

But if you want to use async to parse faster or make a parallel-friendly GUI, you are more than likely wasting your time using python. The benefits will be bottlenecked by other language design features. Still the GIL mostly.

I guess there is no reason you can’t make tokio in python with multiprocessing or subinterpreters, but to my knowledge that hasn’t been done.

Learning tokio was way more fun, too.

Re: Python has had async for 10 years – why isn't it more popular?

#70

Post by xg15 »

I learned about the concept of async/await from JS and back then was really amazed by the elegance of it.

By now, the downsides are well-known, but I think Python's implementation did a few things that made it particularly unpleasant to use.

There is the usual "colored functions" problem. Python has that too, but on steroids: There are sync and async functions, but then some of the sync functions can only be called from an async function, because they expect an event loop to be present, while others must not be called from an async function because they block the thread or take a lot of CPU to run or just refuse to run if an event loop is detected. That makes at least four colors.

The API has the same complexity: In JS, there are 3 primitives that you interact with in code: Sync functions, async functions and promises. (Understanding the event loop is needed to reason about the program, but it's never visible in the code).

Whereas Python has: Generators, Coroutines, Awaitables, Futures, Tasks, Event Loops, AsyncIterators and probably a few more.

All that for not much benefit in everyday situations. One of the biggest advantages of async/await was "fearless concurrency": The guarantee that your variables can only change at well-defined await points, and can only change "atomically". However, python can't actually give the first guarantee, because threaded code may run in parallel to your async code. The second guarantee already comes for free in all Python code, thanks to the GIL - you don't need async for that.

Post reply on HN