How to choose the right Python concurrency API

I always struggle with this. Even though I understand IO-bound and CPU-bound tasks, it's not always one or the other when you build an application. For example, I have to train a machine learning model where I need to query a database or read some files which are considered to be IO bound. But then when I am training a huge model it will become a CPU bound task. How do you go about this?

Would you then separate the logic in such a way that you can use both multi-processing and multi-threading/co-routine. Is this possible when using async? I have limited experience but it feels like the moment you introduce async everything has to be async in the code.

The article gives a good summary of the quite complex landscape of concurrency in python. There's more to it, for example gil-free c-extensions, subprocesses and cross-machine (plus IPC) communication.

But I'm particularly bothered by the fact that many articles and tutorials look at concurrency as if it's only about factoring primes or writing a web server with many (perhaps even idempotent) parallel requests.

In reality, people will often want and need to combine multiple of these approaches, and then it gets VERY messy. I.e. try to combine a multiprocessing executor with multiple asyncio loops and boom you're in some very deep waters.

One project that does this (async loops inside multiple processes) is proxy.py - very enlightening to read its code base [1].

But I really, really wish python would do more to provide simple and robust abstractions for these kinds of tasks. My dream would be a robust actor system similar to erlang, but we'll probably never get that.

[1] https://github.com/abhinavsingh/proxy.py

It's actually frustratingly complex.

Right now I'm working on a "bridge" that receives http requests, and then needs to send that on an existing websocket connection to another system then wait for some responses, send some more on the ws etc and keep monitoring the ws. So the idea is basically to have multiple permanent websocket workers (one for each machine we need to speak with), that get tasks sent to them. Some added complexity is that each machine can only ever have one socket opened at a time.

If I do it using asyncio, I end up with the issue of GIL making it so that I can't really do stuff concurrently, which sucks as the number of incoming requests and machines I need to bridge to increases. Or if I do it using multiple or sub-processes they can no longer communicate, I may risk having multiple processes trying to establish a connection, or processes dying I have to handle manually vs k8s doing it. Or I can do it with different deploys, and having a queue/db or whatever inbetween, drastically increasing complexity.

So hard to combine different modes in the same app.

While in java land I would just have fired up a webserver and made a thread+worker per ws connection and it would've scaled to thousands without issue.

I kind of wish we'd just add Go-style CSP to python and call it a day - it seems to be the best of both worlds when it comes "do two things at the same time".

I didn't see gevent mentioned. Did asyncio make it obsolete?