Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Bringing multithreading to Python's async event loop

github.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Show HN: Bringing multithreading to Python's async event loop

#31

Post by svieira »

bastawhiz wrote: post #23
svieira wrote: post #22

Earlier quoted context omitted.

Looking at the article he's not implementing `Task` with `Thread` - he's round-robinning processing `Task`s through simple `ThreadPool`. So instead of a single `Thread` making continuous progress on the work in the event loop he instead has a set of `Thread`s making progress _in parallel_ on work in the event loop. This is very much Java 21's approach to virtual threads (as well as in-language task runners like the k…

How is that materially different than just making async invocations thread forks and awaits into joins? I understand what the code is doing, I just don't understand what the point is, when it seems like the net effect is the same as just writing threaded code.

The difference is that you can spin up only so many OS threads and you can run several orders of magnitude more "green threads" / "tasks" like this that round-robin onto system threads that comprise your event loop executor. The key thing to understand is that `await` doesn't block the backing thread it simply stops the current task (the backing thread moves on to picking up the next ready task from the queue and running the new task to its next await point).

Re: Show HN: Bringing multithreading to Python's async event loop

#32

Post by adamc »

game_the0ry wrote: post #24
trashtester wrote: post #18

Earlier quoted context omitted.

The main object is the in-between-land, where you need 10x-50x the performance of Python, not 500x the performance on some parallelizable workload. And in many teams, just having to worry about python makes it easier to keep team members productive if they're not expected to handle several different languages productively.

Fair point. Tho I will push back on... > And in many teams, just having to worry about python makes it easier to keep team members productive if they're not expected to handle several different languages productively. I think this makes sense for individuals and teams, but for an org or company I think having specialist teams makes sense where teams that require perf use JVM and teams that make business-ware or devop…

It leads to other problems. A lot of orgs have specifically moved away from specialist teams towards teams united by the business mission.

Re: Show HN: Bringing multithreading to Python's async event loop

#33

Post by game_the0ry »

gloryjulio wrote: post #27
game_the0ry wrote: post #17

I am fairly confident I will get some down votes for this, but here goes... When I am trying to solve a technical problem, the problem is going to dictate my choice of tooling. If I am doing some fast scripting or I need to write some glue code, python is my go-to. But if I have a need for resource efficiency, multi threading, non-blocking async i/o, and/or hi performance, I would not consider python - I would probab…

The problem is that your example is not most of the companies that uses python are facing, which is the majority of the python code. They want some kind of performance uplift without rewriting the whole python code base. It's cheaper if python keeps getting some kind of upgrade An example is facebook's php to hack compiler

Just playing devil's advocate here.

> They want some kind of performance uplift without rewriting the whole python code base.

In order take advantage of mutli-threading and/or async i/o, you would need re-write your code anyway, right? And at the point, wouldn't re-writing in different language be an option?

Re: Show HN: Bringing multithreading to Python's async event loop

#34

Post by biorach »

game_the0ry wrote: post #33
gloryjulio wrote: post #27

Earlier quoted context omitted.

The problem is that your example is not most of the companies that uses python are facing, which is the majority of the python code. They want some kind of performance uplift without rewriting the whole python code base. It's cheaper if python keeps getting some kind of upgrade An example is facebook's php to hack compiler

Just playing devil's advocate here. > They want some kind of performance uplift without rewriting the whole python code base. In order take advantage of mutli-threading and/or async i/o, you would need re-write your code anyway, right? And at the point, wouldn't re-writing in different language be an option?

> you would need re-write your code anyway, right?

Heavily restructure, sure. Rewrite? Probably not.

Re: Show HN: Bringing multithreading to Python's async event loop

#35

Post by gloryjulio »

game_the0ry wrote: post #33
gloryjulio wrote: post #27

Earlier quoted context omitted.

The problem is that your example is not most of the companies that uses python are facing, which is the majority of the python code. They want some kind of performance uplift without rewriting the whole python code base. It's cheaper if python keeps getting some kind of upgrade An example is facebook's php to hack compiler

Just playing devil's advocate here. > They want some kind of performance uplift without rewriting the whole python code base. In order take advantage of mutli-threading and/or async i/o, you would need re-write your code anyway, right? And at the point, wouldn't re-writing in different language be an option?

Not really. The engineering effort of rewriting the entire codebase into a different language is astronomical. Besides mapping the logic to the new language, think about all the quirks between languages that you need to deal with. In the worst cases, you have to come up entirely new code. Nobody wants to pay for that.

Upgrading the language however it's way easier, and you usually have the official upgrade guide about what to do. It's also much safer, easier to deploy and test with.

Once we have the sane multi threading path in python, there would be even less incentive to rewrite the code

Re: Show HN: Bringing multithreading to Python's async event loop

#36

Post by zo1 »

sandeep1998 wrote: post #29
zo1 wrote: post #11

Earlier quoted context omitted.

I've had plenty of success using run_in_executor in production. You basically just use the decorator, and it mostly just works.

Which decorator?

I went back to look at some of the old code as my memory was hazy. It was tornado's implementation of the run_on_executor method that we used, which is used as a decorator.

https://www.tornadoweb.org/en/stable/concurrent.html#tornado...

Re: Show HN: Bringing multithreading to Python's async event loop

#37

Post by maest »

game_the0ry wrote: post #25
mywittyname wrote: post #21

Earlier quoted context omitted.

Sure, this works sometimes. But sometimes you have mountains of code and infrastructure dedicated to one platform, and it's worth the effort to round off the occasional square peg in the interest of operational simplicity/consistency. I've been using ThreadPoolExecutors in Python for a while now. They seem to work pretty well for my use cases. Granted, my use cases don't require things like shared memory segments; I…

> Sure, this works sometimes. But sometimes you have mountains of code and infrastructure dedicated to one platform, and it's worth the effort to round off the occasional square peg in the interest of operational simplicity/consistency. I agree with this, this is a fair trade off, but not the direction I would go as a matter of preference.

What's the alternative, though?

1. Rewrite the whole thing 2. Carve out the high perf component into a separate system and also deal with the overhead of marshalling data between two different systems?

Re: Show HN: Bringing multithreading to Python's async event loop

#38

Post by d0mine »

nbsande wrote: post #7
noident wrote: post #3

> It got me wondering if it was actually possible to make Python’s async event loop work with multiple threads. There is built-in support for this. Take a look at loop.run_in_executor. You can await something scheduled in a separate Thread/ProcessPoolExecutor. Granted, this is different than making the async library end-to-end multi-threaded as you seem to be trying to do, but it does seem worth mentioning in this co…

run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!

run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).

Re: Show HN: Bringing multithreading to Python's async event loop

#39

Post by nbsande »

game_the0ry wrote: post #17

I am fairly confident I will get some down votes for this, but here goes... When I am trying to solve a technical problem, the problem is going to dictate my choice of tooling. If I am doing some fast scripting or I need to write some glue code, python is my go-to. But if I have a need for resource efficiency, multi threading, non-blocking async i/o, and/or hi performance, I would not consider python - I would probab…

Hard agree. If you want resource efficiency and high performance you're probably better off looking to lower level languages most of the time. In my experience FastAPI usually gets used by teams that need a server done quickly and simply or are constrained by a lack of experience in low level languages. That being said, I do think its worthwhile trying to improve efficiency slightly even for these cases.

Re: Show HN: Bringing multithreading to Python's async event loop

#40

Post by nbsande »

gloryjulio wrote: post #27
game_the0ry wrote: post #17

I am fairly confident I will get some down votes for this, but here goes... When I am trying to solve a technical problem, the problem is going to dictate my choice of tooling. If I am doing some fast scripting or I need to write some glue code, python is my go-to. But if I have a need for resource efficiency, multi threading, non-blocking async i/o, and/or hi performance, I would not consider python - I would probab…

The problem is that your example is not most of the companies that uses python are facing, which is the majority of the python code. They want some kind of performance uplift without rewriting the whole python code base. It's cheaper if python keeps getting some kind of upgrade An example is facebook's php to hack compiler

The use case I wrote it in mind with is FastAPI. In that case, there wouldn't be any change to the Python code. You'd just use a different ASGI server that would use this sort of multithreaded event loop. So instead of running it with uvicorn main:app, you'd run it with alternateASGI main:app.

I have an example of a very basic ASGI server that does just that towards the end of the blog

Post reply on HN