Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Bringing multithreading to Python's async event loop

github.com

Post reply on HN
Print view View original

11–20 of 47 posts

Re: Show HN: Bringing multithreading to Python's async event loop

#11

Post by zo1 »

nbsande wrote: post #7
noident wrote: post #3

> It got me wondering if it was actually possible to make Python’s async event loop work with multiple threads. There is built-in support for this. Take a look at loop.run_in_executor. You can await something scheduled in a separate Thread/ProcessPoolExecutor. Granted, this is different than making the async library end-to-end multi-threaded as you seem to be trying to do, but it does seem worth mentioning in this co…

run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!

I've had plenty of success using run_in_executor in production. You basically just use the decorator, and it mostly just works.

Re: Show HN: Bringing multithreading to Python's async event loop

#12

Post by sidmitra »

btown wrote: post #4
quotemstr wrote: post #2

At that point, why bother with asyncio? What we really want is something like Java virtual threads, something that doesn't have a code color.

gevent is exactly this! http://www.gevent.org/ My startup has been using it in production for years. It excels at I/O bound workflows where you have highly concurrent real-time usage of slow/unpredictable partner APIs. You just write normal (non-async) Python code and the patched system internals create yields to the event loop whenever you’d be waiting for I/O, giving you essentially unlimited concurrency (as long a…

I love gevent, but i never was 100% sure that nothing is secretly breaking or some weird thread safety issue. In a large SaaS app all sorts of 3rd party libs do weird background threading stuff or someone randomly starts doing threading.Local and shared global context. After hitting some weird hanging redis-py client issues, i turned gevent off and it went away. Never really got around to spend time to debug the issue(especially since it happened on prod and hard to replicate on stage/local).

Does your app have a lot of dependencies that do background threads? Like Launchdarkly(feature flags), redis, spyne(rpc) and on and on.

Re: Show HN: Bringing multithreading to Python's async event loop

#13

Post by m11a »

Perhaps a silly question, but should an event loop actually be multithreaded?

My understanding was that tasks in an event loop should yield after they dispatch IO tasks, which means the event loop should be CPU-bound right? If so, multithreading should not help much in theory?

Re: Show HN: Bringing multithreading to Python's async event loop

#14

Post by spiffytech »

m11a wrote: post #13

Perhaps a silly question, but should an event loop actually be multithreaded? My understanding was that tasks in an event loop should yield after they dispatch IO tasks, which means the event loop should be CPU-bound right? If so, multithreading should not help much in theory?

If your workload is actually CPU-bound after you've deferred IO to the background, that's exactly when multithreading can help.

I've seen code that spends disproportionate CPU time spent on e.g., JSON (de)serializing large objects, or converting Postgres result sets into native data structures, but sometimes it's just plain ol' business logic. And with enough traffic, any app gets too busy for one core.

Single-threaded langs get around this by deploying multiple copies of the app on each server to use up the cores. But that's less efficient than a single, parallel runtime, and eliminates some architectural options.

Re: Show HN: Bringing multithreading to Python's async event loop

#15

Post by whalesalad »

sidmitra wrote: post #12
btown wrote: post #4

Earlier quoted context omitted.

gevent is exactly this! http://www.gevent.org/ My startup has been using it in production for years. It excels at I/O bound workflows where you have highly concurrent real-time usage of slow/unpredictable partner APIs. You just write normal (non-async) Python code and the patched system internals create yields to the event loop whenever you’d be waiting for I/O, giving you essentially unlimited concurrency (as long a…

I love gevent, but i never was 100% sure that nothing is secretly breaking or some weird thread safety issue. In a large SaaS app all sorts of 3rd party libs do weird background threading stuff or someone randomly starts doing threading.Local and shared global context. After hitting some weird hanging redis-py client issues, i turned gevent off and it went away. Never really got around to spend time to debug the issu…

We also heavily use gevent but this is indeed the greatest frustration. Random and difficult to diagnose issues in external libraries like sockets being closed prematurely or timing out.

Re: Show HN: Bringing multithreading to Python's async event loop

#16

Post by limaoscarjuliet »

m11a wrote: post #13

Perhaps a silly question, but should an event loop actually be multithreaded? My understanding was that tasks in an event loop should yield after they dispatch IO tasks, which means the event loop should be CPU-bound right? If so, multithreading should not help much in theory?

You are correct, async-io is cooperative. This seems an attempt to enhance these async-io cooperative threads work more like goroutines in golang. Golang can start threads if it "thinks" the workload needs more CPU.

Re: Show HN: Bringing multithreading to Python's async event loop

#17

Post by game_the0ry »

I am fairly confident I will get some down votes for this, but here goes...

When I am trying to solve a technical problem, the problem is going to dictate my choice of tooling.

If I am doing some fast scripting or I need to write some glue code, python is my go-to. But if I have a need for resource efficiency, multi threading, non-blocking async i/o, and/or hi performance, I would not consider python - I would probably use JVM over the best python option.

Don't get me wrong, I think its a worthwhile effort to explore this effort, and I certainly do not think its a wasted effort (quite the opposite, this gets my up vote) I just don't think I would ever use it if I had use case for perf and resource efficiency.

Re: Show HN: Bringing multithreading to Python's async event loop

#18

Post by trashtester »

game_the0ry wrote: post #17

I am fairly confident I will get some down votes for this, but here goes... When I am trying to solve a technical problem, the problem is going to dictate my choice of tooling. If I am doing some fast scripting or I need to write some glue code, python is my go-to. But if I have a need for resource efficiency, multi threading, non-blocking async i/o, and/or hi performance, I would not consider python - I would probab…

The main object is the in-between-land, where you need 10x-50x the performance of Python, not 500x the performance on some parallelizable workload.

And in many teams, just having to worry about python makes it easier to keep team members productive if they're not expected to handle several different languages productively.

Re: Show HN: Bringing multithreading to Python's async event loop

#19

Post by bastawhiz »

m11a wrote: post #13

Perhaps a silly question, but should an event loop actually be multithreaded? My understanding was that tasks in an event loop should yield after they dispatch IO tasks, which means the event loop should be CPU-bound right? If so, multithreading should not help much in theory?

This was exactly my question. Why do you even need an event loop? If awaits are just thread joins then what is the event loop actually doing? IO can just block, since other coroutines are on other threads and are unaffected.

Which is to say, why even bother with async if you want your code to be fully threaded? Async is an abstraction designed specifically to address the case where you're dealing with blocking IO on a single thread. If you're fully threaded, the problems async addresses don't exist anymore. So why bother?

Re: Show HN: Bringing multithreading to Python's async event loop

#20

Post by svieira »

quotemstr wrote: post #6
drowsspa wrote: post #5

Earlier quoted context omitted.

Not for those of us stuck in Java 8... Is it stable already?

I hear that some Amish sects permit the use of technology that's older and proven not to be too worldly, like washing machines, chainsaws, and Java 11. Have you considered converting?

Virtual threads only became stable in Java 21, unfortunately (https://openjdk.org/jeps/444). If the issue is "I'm bound to Java 8" proposing running a research build of Java 11 in production is going to fly as well as a lead balloon on the moon.
Post reply on HN