Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Bringing multithreading to Python's async event loop

github.com

Post reply on HN
Print view View original

41–47 of 47 posts

Re: Show HN: Bringing multithreading to Python's async event loop

#41

Post by nbsande »

bastawhiz wrote: post #19
m11a wrote: post #13

Perhaps a silly question, but should an event loop actually be multithreaded? My understanding was that tasks in an event loop should yield after they dispatch IO tasks, which means the event loop should be CPU-bound right? If so, multithreading should not help much in theory?

This was exactly my question. Why do you even need an event loop? If awaits are just thread joins then what is the event loop actually doing? IO can just block, since other coroutines are on other threads and are unaffected. Which is to say, why even bother with async if you want your code to be fully threaded? Async is an abstraction designed specifically to address the case where you're dealing with blocking IO on…

Having too many threads all running at the same time can also cause a performance hit, and I don't mean hitting the OS limit on threads. The more threads you have running in parallel(remember this is considering a GIL-less setup) the more you need to context switch between the. Having fewer threads all running in an event loop allows you to manage more events with only a few threads, for example setting the number of event loop threads to the number of cores on the cpu.

Re: Show HN: Bringing multithreading to Python's async event loop

#42

Post by nbsande »

m11a wrote: post #30
bastawhiz wrote: post #23

Earlier quoted context omitted.

How is that materially different than just making async invocations thread forks and awaits into joins? I understand what the code is doing, I just don't understand what the point is, when it seems like the net effect is the same as just writing threaded code.

If I understand correctly, it sounds like the idea is to map N tasks to M threads. I suppose it’d only really be useful if you have more tasks than you can have OS threads (due to the memory overhead of an OS thread), then maybe 10,000 tasks can run in 16 OS threads. If that’s the case, then is this useful in any application other than when you have way too many threads to feasibly make each task an OS thread?

The idea is to map N tasks to M threads. This is useful more than just when you needd more threads than the OS can spin up. As you scale up the number of threads you increase context switching and cpu scheduling overhead. Being able to schedule A large number of tasks with a small number of threads could reduce this overhead.

Re: Show HN: Bringing multithreading to Python's async event loop

#43

Post by drowsspa »

svieira wrote: post #20
quotemstr wrote: post #6

Earlier quoted context omitted.

I hear that some Amish sects permit the use of technology that's older and proven not to be too worldly, like washing machines, chainsaws, and Java 11. Have you considered converting?

Virtual threads only became stable in Java 21, unfortunately ( https://openjdk.org/jeps/444 ). If the issue is "I'm bound to Java 8" proposing running a research build of Java 11 in production is going to fly as well as a lead balloon on the moon.

Even so, it's pretty new, isn't it? I don't quite trust the claim it's completely transparent to applications and libraries...

Re: Show HN: Bringing multithreading to Python's async event loop

#44

Post by trashtester »

game_the0ry wrote: post #24
trashtester wrote: post #18

Earlier quoted context omitted.

The main object is the in-between-land, where you need 10x-50x the performance of Python, not 500x the performance on some parallelizable workload. And in many teams, just having to worry about python makes it easier to keep team members productive if they're not expected to handle several different languages productively.

Fair point. Tho I will push back on... > And in many teams, just having to worry about python makes it easier to keep team members productive if they're not expected to handle several different languages productively. I think this makes sense for individuals and teams, but for an org or company I think having specialist teams makes sense where teams that require perf use JVM and teams that make business-ware or devop…

JVM based development has a place for some teams. Others need to go all the way to C++/C/Rust etc for the performance they need.

But plenty of tasks can be done beautifully in Python. That's especially true in a data processing or ML setting where most of the heavy lifting is done in libraries such as numpy, spark, pytorch etc. (Also Python is the industry standard for such teams).

Still, even for such teams there are times where you want to do SOME heavier compute tasks within the core language, and offloading this to some other dev team simply doesn't scale.

The solution is to use multi processing instead of multi threading. But this workaround is quite inflexible.

Some dev teams may have developers that can deliver this in scala (especially if spark is involved). Other may have the ability to build C++ (or CUDA) libraries to add to the python environment.

But the ability to run somewhat heavier processing than what can be achieved by a single process is often much better.

Cost wise it also makes a lot of sense. Such steps may often find themselves on some large compute cluster where you have tens or hundreds of processors (or more) available. If a single step in a processing pipeline on such a cluster can be cut from 2 hours to 1 minute, it can be a large saving. Taking it from 1 minute to 10 seconds means a lot less.

Btw, and with all due respect, the part about teams that require perf using JVM doesn't really match my experience. Where I come from, the Java devs tend to produce the slowest code of all, mostly because every step of the processing is serialized/deserialized as microservices talk to each other for each data element.

Even python based code is often faster (sometimes by order of magnitudes). Partly because of cultural differences between teams (the python code, even when exposed as microservices, tend to work with larger blocks of code. And partly because the really is processed in C++ based libraries within python, that still have a significant edge on JVM based code.

Don't get me wrong: Java has a lot of advantages for many types of business applications, where the business logic complexity can be abstracted in well organized and standardized ways. But it's not typically the go-to language when seeking maximum performance in heavy compute or massive data volume scenarios.

Re: Show HN: Bringing multithreading to Python's async event loop

#45

Post by trashtester »

gloryjulio wrote: post #35
game_the0ry wrote: post #33

Earlier quoted context omitted.

Just playing devil's advocate here. > They want some kind of performance uplift without rewriting the whole python code base. In order take advantage of mutli-threading and/or async i/o, you would need re-write your code anyway, right? And at the point, wouldn't re-writing in different language be an option?

Not really. The engineering effort of rewriting the entire codebase into a different language is astronomical. Besides mapping the logic to the new language, think about all the quirks between languages that you need to deal with. In the worst cases, you have to come up entirely new code. Nobody wants to pay for that. Upgrading the language however it's way easier, and you usually have the official upgrade guide abou…

> Not really. The engineering effort of rewriting the entire codebase into a different language is astronomical.

Not to mention that python is actually a good language choice for many types of environments, and basically the industry standard for fields like ML/AI and supporting data pipelines.

Wherever python is used for heavy duty number crunching or large data volumes, most processing is handled by libraries written in other languages, while python is handling program flow and some small parts that need custom code. The large part can currently be quite expensive.

Migrating the whole codebase to another language for such setups would simply be absurd.

Still, for the small percentage of such codebases that DOES do semi-heavy data crunching, real multithreading would be nice so one can avoid resorting to multi-processing or implementing these parts as custom C++ libraries, or similar.

Re: Show HN: Bringing multithreading to Python's async event loop

#46

Post by noident »

d0mine wrote: post #38
nbsande wrote: post #7

Earlier quoted context omitted.

run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!

run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).

>run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code

That isn't true. If you use a ProcessPoolExecutor as the target instead of the default executor, you will use multiple processes in pure Python code.

Re: Show HN: Bringing multithreading to Python's async event loop

#47

Post by d0mine »

noident wrote: post #46
d0mine wrote: post #38

Earlier quoted context omitted.

run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).

>run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code That isn't true. If you use a ProcessPoolExecutor as the target instead of the default executor, you will use multiple processes in pure Python code.

the default executor must be ThreadPoolExecutor. But you are right you can pass ProcessPoolExecutor too.

https://docs.python.org/3/library/asyncio-eventloop.html#asy...

Post reply on HN