Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

161–170 of 364 posts

Re: Async Python is not faster

#161

Post by willseth »

quietbritishjim wrote: post #146
john-radio wrote: post #130

Earlier quoted context omitted.

> Fundamentally, you do not waste "3 billion cpu cycles" waiting 1000ms for an external service. Making alternative use of the otherwise idle CPU is the purpose (and IMO the proper domain of) operating systems. Sure, the operating system can find other things to do with the CPU cycles when a program is IO-locked, but that doesn't help the program that you're in the situation of currently trying to run. > An async imp…

> unlike a Python program that does not use async and could only proceed linearly through its instructions This isn't how it works. While Python is blocked in I/O calls, it releases the GIL so other threads can proceed. (If the GIL were never released then I'm sure they wouldn't have put threading in the Python standard library.) > Python's multiprocessing library is needed to overcome the GIL This is technically tru…

> But blocking IO isn't one of those situations, so you can just use threads.

Threads and async are not mutually exclusive. If your system resources aren't heavily loaded, it doesn't matter, just choose the library you find most appropriate. But threads require more system overhead, and eventually adding more threads will reduce performance. So if it's critical to thoroughly maximize system resources, and your system cannot handle more threads, you need async (and threads).

Re: Async Python is not faster

#163

Post by tijsvd »

wongarsu wrote: post #113
calpaterson wrote: post #105

Earlier quoted context omitted.

I think it is surprising to a lot of people who do take it as read that async will be faster. As I describe in the first line of my article I don't think that people who think async is faster have unreasonable expectations. It seems very intuitive to assume that greater concurrency would mean greater performance - at least one some measure. > When you're dealing with external REST APIs that take multiple seconds to r…

Async IO was in large part a response to "how can my webserver handle xx thousand connections per second" (or in the case of Erlang "how do you handle millions of phone calls at once"). Starting 15 threads to do IO works great, but once you wait for hundreds of things at once the overhead from context switching becomes a problem, and at some point the OS scheduler itself becomes a problem

Not really. At least on Linux, the scheduler is O(1). There is no difference between one process waiting for 10k connections, or 10k processes waiting for 1 each. And there is hardly a context switch either, if all these 10k processes use the same memory map (as threads do).

I've tested this extensively on Linux. There is no more CPU used for threads vs epoll.

On the other hand, if you don't get the epoll imementation exactly right, you may end up with many spurious calls. E.g. simply reading slow data from a socket in golang on Linux incurs considerable overhead: a first read that is short, another read that returns EWOULDBLOCK, and then a syscall to re-arm the epoll. With OS threads, that is just a single call, where the next call blocks and eventually returns new data.

Edit: one thing I haven't considered when testing is garbage collection. I'm absolutely convinced that up to 10k connections, threads or async doesn't matter, in C or Rust. But it may be much harder to do GC over 10k stacks than over 8.

Re: Async Python is not faster

#164

Post by jupp0r »

The general issue if serving one vs multiple clients per thread has been discussed extensively in the last two decades, see http://www.kegel.com/c10k.html

I’m not familiar with python but it seems like there is a glaring performance bug iff using one thread per connection is faster than using async io.

Re: Async Python is not faster

#165

Post by cvlasdkv »

brightball wrote: post #118
calpaterson wrote: post #105

Earlier quoted context omitted.

I think it is surprising to a lot of people who do take it as read that async will be faster. As I describe in the first line of my article I don't think that people who think async is faster have unreasonable expectations. It seems very intuitive to assume that greater concurrency would mean greater performance - at least one some measure. > When you're dealing with external REST APIs that take multiple seconds to r…

This is often a point of confusion for people when looking at Erlang, Elixir or Go code. Concurrency beyond leveraging available CPU's doesn't really add any advantage. On the web when the bulk of your application code time is waiting on APIs, database queries, external caches or disk I/O it creates a dramatic increase in the capacity of your server if you can do it with minimal RAM overhead. It's one of the big reas…

From my understanding the primary use of concurrency in Erlang/Elixir is for isolation and operational consistency. Do you believe that not to be the case?

Re: Async Python is not faster

#166

Post by lovasoa »

Async python is faster when you use it for running parallel tasks. In this benchmark, you are running a single database request per query, so there is no advantage to being asynchronous: a pool of processes will scale just as well (but it will use more memory). The point of async is that it lets you easily make a Postgres query, AND an HTTP query, AND a redis query in parallel.

Re: Async Python is not faster

#168

Post by otabdeveloper4 »

willseth wrote: post #161
quietbritishjim wrote: post #146

Earlier quoted context omitted.

> unlike a Python program that does not use async and could only proceed linearly through its instructions This isn't how it works. While Python is blocked in I/O calls, it releases the GIL so other threads can proceed. (If the GIL were never released then I'm sure they wouldn't have put threading in the Python standard library.) > Python's multiprocessing library is needed to overcome the GIL This is technically tru…

> But blocking IO isn't one of those situations, so you can just use threads. Threads and async are not mutually exclusive. If your system resources aren't heavily loaded, it doesn't matter, just choose the library you find most appropriate. But threads require more system overhead, and eventually adding more threads will reduce performance. So if it's critical to thoroughly maximize system resources, and your system…

> But threads require more system overhead, and eventually adding more threads will reduce performance.

Absolutely false. OS threads are orders of magnitude lighter than any Python coroutine implementation.

Re: Async Python is not faster

#169

Post by anentropic »

orf wrote: post #43

His async code creates a pool with only 10 max connections[1] (the default). Whereas his sync pool[2], with a flask app that has 16 workers, has significantly more database connections. I expect upping this number would have a positive effect on asyncio numbers because the only thing[3] this[4] is[5] measuring[6] is how many database connections you have, and is about as far from a realistic workload as you can get.…

> Change your app to make 3 parallel requests to httpbin, collect the responses and insert them into the database. That's an actually realistic asyncio workload

I don't see how that is a more "realistic" asyncio workload.

It might be a workload that async is better suited for, but the point of the article is to compare async web frameworks, which will often be used just to fetch and return some data from the db.

If you had an endpoint which needed to fetch 3 items from httpbin and insert them in the db it may make sense to use asyncio tools for that, even within the context of a web app running under a sync framework+server like Falcon+Gunicorn.

In my experience Python web apps (Django!) often spend surprisingly little time waiting on the db to return results, and relatively a large amount of time cpu-bound instantiating ORM model instances from the db data, then transforming those instances back into primitive types that can be serialized to JSON in an HTTP response. In that context I am not surprised if sync server with more processes is performing better. In this test it's not even that bad... the 'ORM' seems to be returning just a tuple which is transformed to a dict and then serialized.

Re: Async Python is not faster

#170

Post by Spiritus »

hedora wrote: post #145
Spiritus wrote: post #115

Funny how in the TechEmpower Web Framework Benchmarks[1], the async frameworks are basically destroying their sync counterparts. [1] https://www.techempower.com/benchmarks/

None of the performant async frameworks in those benchmarks are written in python (unless I missed one). The takeaway of this article is that python’s async io implementations perform poorly. That’s surprising, since async I/O is usually used for performance reasons, and in most other languages, async I/O can be much faster.

The idea is to filter so that it only shows Python. Not to compare with other languages.
Post reply on HN