Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

141–150 of 364 posts

Re: Async Python is not faster

#141

Post by calpaterson »

Spiritus wrote: post #115

Funny how in the TechEmpower Web Framework Benchmarks[1], the async frameworks are basically destroying their sync counterparts. [1] https://www.techempower.com/benchmarks/

I think that is largely because they aren't using enough workers. IIRC they use 3 * cpu_count for sync, which just won't be enough.

They also misleadingly de-emphasise latency variation. One Python async framework I'd never heard of was top of the pops on throughput there even though the latency numbers suggested it had pretty much fallen apart in the test.

Re: Async Python is not faster

#142

Post by nerdponx »

zzzeek wrote: post #122

I am SUPER happy someone else is finally looking at this. It is long past time that the reflexive use of asycnio or systems like gevent/eventlet for no other reason than "hand-wavy SPEED" come to an end. That web applications that literally serve just one user at at time are built in Tornado for "speed". (my example for this is the otherwise excellent SnakeViz: https://jiffyclub.github.io/snakeviz/ which IMO should h…

This is kind of weird to me though. All this effort being spent to argue what is effectively a strawman belief only held by people who don't fully understand what they believe.

Re: Async Python is not faster

#143

Post by brodouevencode »

Great article, but don't just abandon async entirely. There are still use cases. For me: I use it to pull data from several external APIs all at once. That data is then married up to produce another data object with some special sauce computation. All of these network calls run in parallel, therefore the network overhead (DNS lookups, SSL handshakes, etc.) all operate at the same time instead of running one after the other if it were in synchronous mode. IIRC the benchmarks for this went from running at 3 minutes+ to just over 20 seconds.

So there's still utility, so YMMV.

Re: Async Python is not faster

#144

Post by _pmf_ »

calpaterson wrote: post #105
phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

I think it is surprising to a lot of people who do take it as read that async will be faster. As I describe in the first line of my article I don't think that people who think async is faster have unreasonable expectations. It seems very intuitive to assume that greater concurrency would mean greater performance - at least one some measure. > When you're dealing with external REST APIs that take multiple seconds to r…

> I think it is surprising to a lot of people who do take it as read that async will be faster.

Literally the first thing any concurrency course starts with in the very first lesson is that scheduling and context overhead are not negligible. Is it so hard to expect our professionals to know basic principles of what they are dealing with?

Re: Async Python is not faster

#145

Post by hedora »

Spiritus wrote: post #115

Funny how in the TechEmpower Web Framework Benchmarks[1], the async frameworks are basically destroying their sync counterparts. [1] https://www.techempower.com/benchmarks/

None of the performant async frameworks in those benchmarks are written in python (unless I missed one).

The takeaway of this article is that python’s async io implementations perform poorly.

That’s surprising, since async I/O is usually used for performance reasons, and in most other languages, async I/O can be much faster.

Re: Async Python is not faster

#146

Post by quietbritishjim »

john-radio wrote: post #130
calpaterson wrote: post #105

Earlier quoted context omitted.

I think it is surprising to a lot of people who do take it as read that async will be faster. As I describe in the first line of my article I don't think that people who think async is faster have unreasonable expectations. It seems very intuitive to assume that greater concurrency would mean greater performance - at least one some measure. > When you're dealing with external REST APIs that take multiple seconds to r…

> Fundamentally, you do not waste "3 billion cpu cycles" waiting 1000ms for an external service. Making alternative use of the otherwise idle CPU is the purpose (and IMO the proper domain of) operating systems. Sure, the operating system can find other things to do with the CPU cycles when a program is IO-locked, but that doesn't help the program that you're in the situation of currently trying to run. > An async imp…

> unlike a Python program that does not use async and could only proceed linearly through its instructions

This isn't how it works. While Python is blocked in I/O calls, it releases the GIL so other threads can proceed. (If the GIL were never released then I'm sure they wouldn't have put threading in the Python standard library.)

> Python's multiprocessing library is needed to overcome the GIL

This is technically true, in that if you are running up against the GIL then the only way to overcome it is to use multiprocessing. But blocking IO isn't one of those situations, so you can just use threads.

The comparison here is not async vs just doing one thing. It's async vs threads. I believe that's what the performance comparison in the article is about, and if threads were as broken as you say then obviously they wouldn't have performed better than asyncio.

--------

As an aside, many C-based extensions also release the GIL when performing CPU-bound computations e.g. numpy and scipy. So GIL doesn't even prevent you from using multithreading in CPU-heavy applications, so long as they are relatively large operations (e.g. a few calls to multiply huge matrices together would parallelise well, but many calls to multiply tiny matrices together would heavily contend the GIL).

Re: Async Python is not faster

#147

Post by kerkeslager »

phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

> The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O.

This is a quintessential example of not seeing the forest for the trees.

The point of coroutines is absolutely to make my code execute faster. If a completely I/O-bound application sits idle while it waits for I/O, I don't care and I should not care because there's no business value in using those wasted cycles. The only case where coroutines are relevant is when the application isn't completely I/O bound; the only case where coroutines are relevant is when they make your code execute faster.

It's been well-known for a long time that the majority of processes in (for example) a webserver, are I/O bound, but there are enough exceptions to that rule that we need a solution to situations where the process is bound by something else, i.e. CPU. The classic solution to this problem is to send off CPU-bound processes to a worker over a message queue, but that involves significant overhead. So if we assume that there's no downside to making everything asynchronous, then it makes sense to do that--it's not faster for the I/O bound cases, but it's not slower either, and in the minority-but-not-rare CPU-bound case, it gets us a big performance boost.

What this test is doing is challenging the assumption that there's no downside to making everything asynchronous.

In context, I tend to agree with the conclusion that there are downsides. However, those downsides certainly don't apply to every project, and when they do, there may be a way around them. The only lesson we can draw from this is that gaining benefit from coroutines isn't guaranteed or trivial, but there is much more compelling evidence for that out there.

Re: Async Python is not faster

#148

Post by 1337shadow »

mattbillenstein wrote: post #109
PaulHoule wrote: post #101

Earlier quoted context omitted.

I've written Python functions that "call" another function either async or not depending on how the function inspects. For instance, imagine a "maybe_await" method that just calls sync if is synchronous or otherwise awaits.

It's very heavy to do this is it not? Like you inspect the function on each call to figure out if it's async or not?

You don't inspect the function, but the result. Is it a coroutine ? If so, it needs to be awaited (see my example above).

I believe this also would allow non-async function to return a coroutine I suppose.

Anyway, in this case chances are that there will be no performance overhead if there's any io bound operation running in the coroutine so it should run the iscoroutine check of the result and the await call before the function is done.

Re: Async Python is not faster

#150

Post by Grimm1 »

Async is useful for high IO where you may have a lot of down time between the requests. Are you pulling many requests from different servers with different response times, communicating with a db or pulling out large response bodies. Async is probably going to do better since each one of those synchronously represents potentially large idling periods where other requests could have gotten work done.

As to the article the comparisons are good but fails to mention resource constraints, like Gunicorn, forking 16 instances is going to be a lot heavier on memory so for a little more RPS you're probably spending a decent chunk of change more to run your work and I don't think that's worth it considering the Async model in python is pretty easy to grok these days and under this benchmark share a similar performance profile.

Now that said If I had to guess these numbers are fine for the average API but if you're doing something like high throughput web crawling or need to serve something on the order of 10's of thousands to hundred thousands RPS async will win out on speed and resource use and ultimately cost.

Plus at one point they were like "we could only get an 18% speed up with Vibora" haven't used them my self. But 18% performance increase at really any level of load is fantastic. Hand waving that off tells me the work loads for what is "realistic" don't take in to account real high RPS workloads like you might see at major tech companies.

Post reply on HN