Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python has had async for 10 years – why isn't it more popular?

tonybaloney.github.io

Post reply on HN
Print view View original

51–60 of 305 posts

Re: Python has had async for 10 years – why isn't it more popular?

#51

Post by gen220 »

atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

As somebody who's written and maintained a good bit of Python in prod and recently a good amount of server-side typescript... this would be my answer.

I'd add one other aspect that we sort of take for granted these days, but affordable multi-threaded CPUs have really taken off in the last 10 years.

Not only does the stack based on green-threads "just work" without coloring your codebase with async/no-async, it allows you to scale a single compute instance gracefully to 1 instance with N vCPUs vs N pods of 2-vCPU instances.

Re: Python has had async for 10 years – why isn't it more popular?

#52

Post by kibwen »

throw-qqqqq wrote: post #30
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…

> Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies

But this appears to be describing languages with green threads, rather than languages that make async explicit.

Re: Python has had async for 10 years – why isn't it more popular?

#53

Post by ronsor »

JackSlateur wrote: post #42
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

green thread have pitfalls too, like this: https://news.ycombinator.com/item?id=39008026

This doesn't look like a problem with green threads so much as it is a problem with Java's implementation of them. Of course, Java is known for having problems with its implementations of many different things, such as sandboxing; this isn't special.

Re: Python has had async for 10 years – why isn't it more popular?

#54

Post by lacker »

throw-qqqqq wrote: post #30
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…

I'm confused, I feel like the two of you are expressing opposite opinions.

The comment you are responding to prefers green threads to be managed like goroutines, where the code looks synchronous, but really it's cooperative multitasking managed by the runtime, to explicit async/await.

But then you criticize "code that looks synchronous but is really async". So you prefer the explicit "async" keywords? What exactly is your preferred model here?

Re: Python has had async for 10 years – why isn't it more popular?

#55

Post by rdtsc »

I never liked async in Python. I feel like it's a bad design pattern, a lot of borrowed from Twisted at the time. I always liked gevent/eventlet based approach and will likely always stick to using that. At the time Go and Elixir/Erlang had green threads (lightweight procs / goroutines) and in general I think that makes for a cleaner code base.

Re: Python has had async for 10 years – why isn't it more popular?

#56

Post by whilenot-dev »

MichaelRazum wrote: post #8

You can’t just plug and play it. As soon as you introduce async you need to have the runtime loop and so on. Basically the whole architecture needs to be redesigned

asyncio has been designed to be as "plug and play" as it gets. I'd discourage it, but one could create async loops wherever one would need them, one separate thread per loop, and adapt the code base in a more granular fashion. Blocking through the GIL will persist, though.

For any new app that is mostly IO constraint I'd still encourage the use of asyncio from the beginning.

Re: Python has had async for 10 years – why isn't it more popular?

#57

Post by cyberax »

To add to this, async in Python is also plain buggy. For example, uvicorn is the most popular FastAPI server, and it leaks contexts across requests in the default installation.

The bug has been open for 2 years, with zero fucks given. The workaround is "just use libuv": https://github.com/encode/uvicorn/issues/2167

I've seen other such cases, and I just gave up on trying to use async.

Re: Python has had async for 10 years – why isn't it more popular?

#58

Post by pclmulqdq »

kibwen wrote: post #52
throw-qqqqq wrote: post #30

Earlier quoted context omitted.

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…

> Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies But this appears to be describing languages with green threads, rather than languages that make async explicit.

Without the "async" keyword, you can still write async code. It looks totally different because you have to control the state machine of task scheduling. Green threads are a step further than the async keyword because they have none of the function coloring stuff.

You may think of use of an async keyword as explicit async code but that is very much not the case.

If you want to see async code without the keyword, most of the code of Linux is asynchronous.

Re: Python has had async for 10 years – why isn't it more popular?

#59

Post by larusso »

throw-qqqqq wrote: post #30
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances. I take so much flak for this opinion at work, but I agree with you 100%. Code that looks synchronous, but is really async, has funny failure modes and idiosyncracies, and I generally see more bugs in the async parts of our code at work. Maybe I’m just old, but I do…

async is like a virus. I think the implementation in js and .NET is somewhat ok’ish because your code is inside an async context most of the time. I really hate the red / blue method issues where library functions get harder to compose. Oh I have a normal method because there was no need for async. Now I change the implementation and need to call an async method. There are ways around this but more often than not will you change most methods to be async.

To be fair that also happens with other solutions.

Re: Python has had async for 10 years – why isn't it more popular?

#60

Post by jayd16 »

atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

> But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances.

I'll be sold on this when a green thread native UI paradigm becomes popular but it seems like all the languages with good native UI stories have async support.

Post reply on HN