Live data from Hacker News

Python has had async for 10 years – why isn't it more popular?

tonybaloney.github.io

71–80 of 305 posts

Re: Python has had async for 10 years – why isn't it more popular?

#71

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

Because it sucks compared to gevent (green threads). But for some reason, people always disregard this option. They don't even read it. Like any comment with gevent is shadowbanned and it doesn't register in their mind.

Happily using gevent for our backend (IoT+ML) since 2019. Was very glad when I saw it is still being well supported by recent SQLAlchemy and pscycopg releases.

Re: Python has had async for 10 years – why isn't it more popular?

#72
post #32

idk stackless dates back to 2005 at least, most likely earlier. greenlet which is sort of minimal stackless .. before 2008 pycoev which is on one hand greenlets without memmove()s, on the other hand sort of io-scheduled m:n threading I wrote myself in 2009. so, at least idk, 20 years? It was first needed. Then 10 years passed, people got around to pushing it through the process aaand by the time it was done it was al…

I never understood why stackless wasn't more popular. It was rather nice, clean and performant (well, it's still python but it provide(d) proper concurrency)

Re: Python has had async for 10 years – why isn't it more popular?

#73
post #70

I learned about the concept of async/await from JS and back then was really amazed by the elegance of it. By now, the downsides are well-known, but I think Python's implementation did a few things that made it particularly unpleasant to use. There is the usual "colored functions" problem. Python has that too, but on steroids: There are sync and async functions, but then some of the sync functions can only be called f…

> some of the sync functions can only be called from an async function, because they expect an event loop to be present

I recognise that this situation is possible, but I don't think I've ever seen it happen. Can you give an example?

Re: Python has had async for 10 years – why isn't it more popular?

#74

You can’t just plug and play it. As soon as you introduce async you need to have the runtime loop and so on. Basically the whole architecture needs to be redesigned

It's this - asyncio is a nightmare to add to and get working in a code base not specifically designed for it, and most people aren't going to bother with that. asyncio is not good enough at anything it does to make it worth it to me to design my entire program around it.

Re: Python has had async for 10 years – why isn't it more popular?

#75

Yes, this!. Its a mess, some typing, some async. No standardization/ one way to do things. Literally goes against the original ZEN of python "There should be one-- and preferably only one --obvious way to do it : Aim for a single, clear solution to a problem. "

Cue the zen of python apologists explaining how we just don't get it and that with enough reframing it'll click. Snide remark aside, I actually like the Zen of Python as programming language folklore but in 2025 AD it's kinda crazy to pretend that Python actually adheres to those tenets or whatever you wish to call them, and I'd go as far as to claim that it does a disservice to a language flexible enough for a lot o…

[deleted]

Re: Python has had async for 10 years – why isn't it more popular?

#76
post #69

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

For me, once I wanted to scale asyncio within one process (scaling horizontally on top of that), only two things made sense: rust with tokio or node.js. Doing async in python has the same fundamental design. You have an executer, a scheduler, and event-driven wakers on futures or promises. But you’re doing it in a fundamentally hand-cuffed environment. You don’t get benefits like static compilation, real work-stealin…

Or just golang?

Re: Python has had async for 10 years – why isn't it more popular?

#78
It was supposed to bring massive concurrency to Python. But as with any async implantation in any language it is too easy to deadlock the entire system. Did you forgot to sprinkle enough `await`? Your code is blocked somewhere, good luck hunting for it.

In contrast preemptive green threads are too easy. Be it IO or CPU load all threads will get their slice of CPU time. Nothing is blocked so you can debug your logic errors instead of deadlocks everywhere.

Async works in JS so well because the entire language is designed for it, instead of async being just bolted on. You can't even run plain `sleep` to block, you need setTimeout.

Re: Python has had async for 10 years – why isn't it more popular?

#79

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

You make a very good case for why python's async isn't more prevalent, but I think this is not painting the full image.

Taking a general case, let's say a forum, in order to render a thread one needs to search for all posts from that thread, then get all the extra data needed for rendering and finally send the rendered output to the client.

In the "regular" way of doing this, one will compose a query, that will filter things out, join all the required data bla bla, send it to the database, wait for the answer from the database and all the data to be transferred over, loop over the results and do some rendering and send the thing over to the client.

It doesn't matter how async your app code is, in this way of doing things, the bottle neck is the database, as there is a fixed limit on how many things a db server can do at once and if doing one of these things takes a long time, you still end up waiting too much.

In order for async to work, one needs to split the work load into very small chunks that can be done in parallel and very fast, therefore, sending a big query and waiting for all the result data is out of the window.

An async approach would split the db query into a search query, that returns a list of object ids, say posts, then create N number of async tasks that given a post id will return a rendered result. These tasks will do their own query to retrieve the post data, then assemble another list of async tasks to get all the other data required and render each chunk and so on. Throw in a bunch of db replicas and you get the benefits of async.

This approach is not generally used, because, let's face it, we like making the systems we use do complicated things, eg complicated sql requests.

Post reply on HN