Not popular?
I use async all the time.
The evidence this post provides is that flask and Django aren’t all in on async.
That’s meaningless.
Python has had async for 10 years – why isn't it more popular?
91–100 of 305 posts
Re: Python has had async for 10 years – why isn't it more popular?
#92Earlier quoted context omitted.
Without the "async" keyword, you can still write async code. It looks totally different because you have to control the state machine of task scheduling. Green threads are a step further than the async keyword because they have none of the function coloring stuff. You may think of use of an async keyword as explicit async code but that is very much not the case. If you want to see async code without the keyword, most…
Having to put "await" everywhere is very explicit. I'd even say it's equally explicit to a bunch of awkward closures. Why do you say it's less?
> Why do you say it's less
Let me try to clarify my point of view:
I don’t mean that async/await is more or less explicit than goroutines. I mean regular threaded code is more explicit than async/await code, and I prefer that.
I see colleagues struggle to correctly analyze resource usage for instance. Someone tries to parallelize some code (perhaps naiively) by converting it to async/await and then run out of memory.
Again, I don’t mean to judge anyone. I just observe that the async/await-flavor has more bugs in the code bases I work on.
Re: Python has had async for 10 years – why isn't it more popular?
#93The barrier it places in Rust's lib ecosystem is unpleasant; I'm glad it hasn't taken off in Python. I have written too many rust libs because the existing ones forced your code to be Async.
Re: Python has had async for 10 years – why isn't it more popular?
#94The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…
"Then java did it too." Java had green threads in 1.0. They were removed. Then Java added virtual threads.
Re: Python has had async for 10 years – why isn't it more popular?
#95Not too long ago, I read a comment on HN that suggested, due to Python's support for free-threading, async in Python will no longer be needed and will lose out to free-threading due to it's use of "colored" functions. Which seems to align with where this author ends up: > Because parallelism in Python using threads has always been so limited, the APIs in the standard library are quite rudimentary. I think there is an…
Interesting that very few people in that thread seem to understand Go's model, especially the author of this proposal. If you don't allow preemption, you still have a sort of coloring because most non async functions aren't safe to call in a virtual thread - they may block the executor. If you call C code, you need to swap out stacks and deal with blocking by potentially spawning more OS threads - that's what CGo doe…
I can't speak to the more technical aspects you bring up b/c I'm not that well versed in the underlying implementations and tradeoffs.
> and also too little too late.
I think it very likely that Python will still be around and popular 10 years from now. Probably 20 years from now. And maybe 30 years from now. I think that's plenty of time for a new and good idea that addresses significant pain points to take root and become a predominant paradigm in the ecosystem.
So I don't agree that it's too little too late. But whether or not a Virtual Threads implementation can/will be developed and be good enough to gain wide adoption, I just can't speak to. If it's possible to create a better devx than async and get multi-core performance and usage, I'm all for the effort.
Re: Python has had async for 10 years – why isn't it more popular?
#96For a counter-opinion that isn't getting stated much here, I think:
* Asyncio is pretty good, and is usually the best choice for non-blocking I/O in python these days.
* Asyncio doesn't add multi-core scaling to python. It's not a replacement for threads and doesn't lift the GIL-imposed scaling limitations. If these things are what you're after from asyncio you'll be disappointed, but they're not what it's trying to add and not adding them doesn't make it a failure.
* "Coloured functions" is a nonsense argument and that article made the whole world slightly more dumb.
* The GIL is part of the reason for python's success. I hope nogil either somehow manages to succeed without compromising the benefits the GIL has brought (I'll be amazed if that happens) or fails entirely. Languages are tools and every tool in your toolbox doesn't have to eventually turn into a drill. If your use case requires in-process parallelisation of interpreted CPU-bound workloads across multiple cores, python is just the wrong thing to use.
* It is indeed extremely annoying that we don't have async file access yet. I hope we get it soon.
Re: Python has had async for 10 years – why isn't it more popular?
#97Earlier quoted context omitted.
Without the "async" keyword, you can still write async code. It looks totally different because you have to control the state machine of task scheduling. Green threads are a step further than the async keyword because they have none of the function coloring stuff. You may think of use of an async keyword as explicit async code but that is very much not the case. If you want to see async code without the keyword, most…
Having to put "await" everywhere is very explicit. I'd even say it's equally explicit to a bunch of awkward closures. Why do you say it's less?
It's explicit that the code is async, but how the asynchrony happens is completely implicit with async/await, and is managed by a runtime of some kind.
Kernel-style async code, where everything is explicit:
* You write a poller that opens up queues and reads structs representing work
* Your functions are not tagged as "async" but they do not block
* When those functions finish, you explicitly put that struct in another queue based on the result
Async-await code, where the runtime is implicit:
* All async functions are marked and you await them if they might block
* A runtime of some sort handles queueing and runnability
Green threads, where all asynchrony is implicit:
* Functions are functions and can block
* A runtime wraps everything that can block to switch to other local work before yielding back to the kernel
Re: Python has had async for 10 years – why isn't it more popular?
#98idk stackless dates back to 2005 at least, most likely earlier. greenlet which is sort of minimal stackless .. before 2008 pycoev which is on one hand greenlets without memmove()s, on the other hand sort of io-scheduled m:n threading I wrote myself in 2009. so, at least idk, 20 years? It was first needed. Then 10 years passed, people got around to pushing it through the process aaand by the time it was done it was al…
Also back then multicore wasn't as prevalent, it made sense to multiplex a zillion things onto one CPU process. Whereas now servers have hundreds of cores / SMT vCPUs [1] and running a lot of processes makes much more sense.
[1] https://www.tomshardware.com/pc-components/cpus/amd-announce...
Re: Python has had async for 10 years – why isn't it more popular?
#99I suppose my negative experiences with async fall under #3, that it is hard to maintain two APIs. One of the most memorable "real software engineering" bugs of my career involved async Python. I was maintaining a FastAPI server which was consistently leaking file descriptors when making any outgoing HTTP requests due to failing to close the socket. This manifested in a few ways: once the server ran out of available f…
> it is hard to maintain two APIs.
This point doesn't get enough coverage. When I saw async coming into Python and C# (the two ecosystems I was watching most closely at the time) I found it depressing just how much work was going into it that could have been productively expended elsewhere if they'd have gone with blocking calls to green threads instead.
To add insult to injury, when implementing async it seems inevitable that what's created is a bizarro-world API that mostly-mirrors-but-often-not-quite the synchronous API. The differences usually don't matter, until they do.
So not only does the project pay the cost of maintaining two APIs, the users keep paying the cost of dealing with subtle differences between them that'll probably never go away.
> I do not prefer Python for reliable, high-performance HTTP servers
I don't use it much anymore, but Twisted Matrix was (is?) great at this. Felt like a superpower to, in the oughties, easily saturate a network interface with useful work in Python.
Re: Python has had async for 10 years – why isn't it more popular?
#100The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…
I also think asyncio missed the mark when it comes to it's API design. There are a lot of quirks and rough edges to it that, as someone who was using `gevent` heavily before, strike me as curious and even anti-productive.