Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python has had async for 10 years – why isn't it more popular?

tonybaloney.github.io

Post reply on HN
Print view View original

211–220 of 305 posts

Re: Python has had async for 10 years – why isn't it more popular?

#211

Post by mdaniel »

stackskipton wrote: post #141
rybosome wrote: post #26

I suppose my negative experiences with async fall under #3, that it is hard to maintain two APIs. One of the most memorable "real software engineering" bugs of my career involved async Python. I was maintaining a FastAPI server which was consistently leaking file descriptors when making any outgoing HTTP requests due to failing to close the socket. This manifested in a few ways: once the server ran out of available f…

Glad I'm not only one in the boat. We have Python HTTP Server doing similar. No one can figure it out, Containerd occasionally OOM kills it, everyone just shrugs and move on.

that tracks so much with my experience in the whole of the python community

Re: Python has had async for 10 years – why isn't it more popular?

#212

Post by dagenix »

The problem, IMO, with asyncio is that its way, way too complicated. In my experience, anyio (https://github.com/agronholm/anyio) provides a much better interface on top of asyncio. And since it can use asyncio as a backend, it maintains compatibility with the asyncio ecosystem. FastAPI, for example, uses anyio.

One thing that I don't see being mentioned in any of the threads here talking about green threads is cancellation. A huge benefit, IMO, of anyio is that it makes cancellation really easy to handle. With asyncio, cancellation is pretty hard. And with green threads, cancellation is often impossible.

Re: Python has had async for 10 years – why isn't it more popular?

#213

Post by ch4s3 »

jacquesm wrote: post #200
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

There are much better solutions for the same problems, but not in Python. If you really need such high throughput you'd move to Go, the JVM or Erlang/Elixer depending on the kind of workload you have rather than to much around with Python on something that it clearly was never intended to do in the first place. It is amazing they got it to work as well as it does but the impedance mismatch is pretty clear and it will…

Elixir is a really nice replacement for a lot of places where you could you python but don't absolutely have to, particularly anything web related. You get a lot more out of the same machine with code that's similarly readable for building HTTP APIs.

Re: Python has had async for 10 years – why isn't it more popular?

#214

Post by mdaniel »

AtlasBarfed wrote: post #135
dekhn wrote: post #38

It added intrusive- codebase-wide- functionality that more or less could have been done with other (thread-based) approaches. AWSCLI was broken for over a year- we had to do a ton of work to deal with the various packaging issues. Don't break userspace.

Why was awscli written in python? Bad decision to begin with. Hey! We have a product that we clearly want to release worldwide. Let's build it on something that doesn't have Unicode. Or any real threading. And is slow as hell. You picked a platform that was going to have to break user space. At least it wasn't JavaScript

I guess at least they're consistent (or maybe herd mentality) since both azure-cli and gcloud are both in python, too

Re: Python has had async for 10 years – why isn't it more popular?

#215

Post by pclmulqdq »

ForHackernews wrote: post #165
throwawayffffas wrote: post #148

Earlier quoted context omitted.

Because it hides away the underlying machinery. Everything is in a run loop that does not exist in my codebase. The context switching points are obvious but the execution environment is opaque. At least that's how it looks to me.

I don't understand this criticism. The JVM is opaque, App Engine is opaque, Docker is opaque. All execution environments are opaque unless you've attached a debugger and are manually poking at the thing while it runs.

Some are more opaque than others.

Re: Python has had async for 10 years – why isn't it more popular?

#216

Post by 0xbadcafebee »

Because it's a niche? You don't need async for most stuff Python is used for, it's a "nice-to-have", and it's annoying to add. If you have to have concurrency/threading/etc, there are other languages with better paradigms.

The same thing happened with Perl and its weird threading (for different reasons, but still)... I guess Python didn't learn that lesson. Perl also gained async and coroutine support, but I think they were added a while after I left the community. I doubt many people use them today. Anyone used them and can comment on ease vs Python?

Re: Python has had async for 10 years – why isn't it more popular?

#217

Post by mdaniel »

fzzzy wrote: post #187
AtlasBarfed wrote: post #135

Earlier quoted context omitted.

Why was awscli written in python? Bad decision to begin with. Hey! We have a product that we clearly want to release worldwide. Let's build it on something that doesn't have Unicode. Or any real threading. And is slow as hell. You picked a platform that was going to have to break user space. At least it wasn't JavaScript

Doesn’t have Unicode? What the heck?

I believe they're discussing that awscli was targeting Python 2.x for the longest time, e.g. https://github.com/aws/aws-cli/blob/0.4.1/setup.py#L48 up through https://github.com/aws/aws-cli/blob/1.19.112/setup.py#L64 from 4 years ago, seemingly cutting over to Py3-only starting with https://github.com/aws/aws-cli/blob/1.20.0/setup.py#L62

I didn't go digging into it, but I'd guess they used the ubiquitous "six" library for backporting unicode functionality, but the point is likely "but why start underwater?!"

Re: Python has had async for 10 years – why isn't it more popular?

#218

Post by markandrewj »

Yoric wrote: post #163
markandrewj wrote: post #139

Earlier quoted context omitted.

I can tell you guys work with languages like Go, so this isn't true for yourselves, but I usually find it is developers that only ever work with synchronous code who find async complicated. Which isn't surprising, if you don't understand something it can seem complicated. My views is almost that people should learn how to write async code by default now. Regardless of the language. Writing modern applications basical…

Hey, I'm one of the (many, many) people who made async in JavaScript happen and I find async complicated.

Hey Yoric, I do not want to underplay what it is like to work with async, but I think there has been a lot of improvements to make it easier, especially in JavaScript/ECMAScript. It is nice not to have to work directly with promises in the same way that was required previously. The language has matured a lot since I started using in Netscape Navigator (I see you formerly worked at Mozilla). I think coding can be complicated in general, although it shouldn't have to be. I think having a mental model for async from the start can be helpful, and understanding the difference between blocking and non blocking code. A lot of people learned writing synchronous code first, so I think it can be hard to develop the mental model and intuit it.

Re: Python has had async for 10 years – why isn't it more popular?

#219

Post by breatheoften »

The problem with python's async is asyncio ...

Structured concurrency libraries like anyio or trio are actually pretty nice -- "stacks" and stack traces are good things. Python multi exception concept is weird --- but also I think probably good ish.

It is still a pita to orchestrate around the gil/how terrible python multiprocessing side effects are wherever cpu bound workloads actually exist ...

Re: Python has had async for 10 years – why isn't it more popular?

#220

Post by mdaniel »

AtlasBarfed wrote: post #135
dekhn wrote: post #38

It added intrusive- codebase-wide- functionality that more or less could have been done with other (thread-based) approaches. AWSCLI was broken for over a year- we had to do a ton of work to deal with the various packaging issues. Don't break userspace.

Why was awscli written in python? Bad decision to begin with. Hey! We have a product that we clearly want to release worldwide. Let's build it on something that doesn't have Unicode. Or any real threading. And is slow as hell. You picked a platform that was going to have to break user space. At least it wasn't JavaScript

but, hey, at least they get to be ultimately dynamic, leading to horeshit like this https://github.com/aws/aws-cli/blob/2.28.22/awscli/botocore/...

What does `create_client` return?! don't you worry your pretty head about it, it'll be whatever you want it to be! flexability!!11

Post reply on HN