Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

71–80 of 364 posts

Re: Async Python is not faster

#71

Post by bildung »

orf wrote: post #43

His async code creates a pool with only 10 max connections[1] (the default). Whereas his sync pool[2], with a flask app that has 16 workers, has significantly more database connections. I expect upping this number would have a positive effect on asyncio numbers because the only thing[3] this[4] is[5] measuring[6] is how many database connections you have, and is about as far from a realistic workload as you can get.…

> His async code creates a pool with only 10 max connections[1] (the default). Whereas his sync pool[2], with a flask app that has 16 workers, has significantly more database connections.

And the reasoning is explained in the article:

"The rule I used for deciding on what the optimal number of worker processes was is simple: for each framework I started at a single worker and increased the worker count successively until performance got worse."

Re: Async Python is not faster

#72

Post by supermatt »

calpaterson wrote: post #59
orf wrote: post #43

His async code creates a pool with only 10 max connections[1] (the default). Whereas his sync pool[2], with a flask app that has 16 workers, has significantly more database connections. I expect upping this number would have a positive effect on asyncio numbers because the only thing[3] this[4] is[5] measuring[6] is how many database connections you have, and is about as far from a realistic workload as you can get.…

Hi - as mentioned in the article all connections went through pgbouncer (limited to 20) and I was careful to ensure that all configurations saturated the CPU so I'm pretty confident they were not waiting on connections to open. Opening a connection from pgbouncer over a unix socket is very fast indeed - my guess is perhaps a couple of orders of magnitude faster than without it. 20 connections divided by 4 CPUs is a l…

Aren't you still capping the throughput by the query rate of your connection pool though? By limiting that, you are limiting the application as a whole - i.e. your benchmark is bound by the speed of your database, and has (almost) nothing to do with the performance of a specific python implementation.

Re: Async Python is not faster

#73

Post by barrkel »

arethuza wrote: post #68
heipei wrote: post #63

Earlier quoted context omitted.

Yeah no, I get that, but what I'm saying is that with sync I can handle as many requests concurrently as I have processes / OS threads running, which is usually the number of cores in my system. With async, I can have thousands of requests in flight, all of them just waiting for the response of the backend, and all I have to do is to start a single OS thread.

In a purely sync world you can have tens of threads per processes without much difficulty - so definitely much heavier resource wise than async but not that bad.

10s?

There are over 2000 threads running on my Linux laptop, with a couple of database servers, an IDE and two browsers open. Firefox has 323 threads, Chrome 420, and Slack 69.

Async code with continuations and no second execution of continuations is isomorphic to threaded code. The chief differences are that in threaded code, the pointer to the continuation function and its closure are on the stack instead of on the heap, and context switches are done by the OS (pro: fairness, con: context switching overhead). Stack allocation and deallocation is generally faster than heap allocation, but because it's contiguous you need to pay for the high water mark. Even then, that's not expensive unless you have loads of recursion or locally bound state.

Re: Async Python is not faster

#74

Post by hombre_fatal »

One big difference between one thread per request vs single-threaded async code is that synchronization and accessing shared resources is trivial when all of your code is running on a single thread.

An entire category of data races like `x += 1` become impossible without you even thinking about it. And that's often worth it for something like a game server where everything is beating on the same data structures.

I don't use Python, so I guess it's less of an issue in Python since you're spawning multiple processes rather than multiple threads so you're already having to share data via something out of process like Redis and using its own synchronization guarantees.

But for example the naive Go code I tend to read in the wild always has data races here and there since people tend to never go 100% into a channel / mutex abstraction (and mutexes are hard). And that's not a snipe at Go but just a reminder of how easy it is to take things for granted when you've been writing single-threaded async code for a while.

Re: Async Python is not faster

#75

Post by kissgyorgy »

orf wrote: post #43

His async code creates a pool with only 10 max connections[1] (the default). Whereas his sync pool[2], with a flask app that has 16 workers, has significantly more database connections. I expect upping this number would have a positive effect on asyncio numbers because the only thing[3] this[4] is[5] measuring[6] is how many database connections you have, and is about as far from a realistic workload as you can get.…

Before you criticize the article, you should read it. He wrote a whole section about the specific worker numbers and why and how he choose them.

Re: Async Python is not faster

#76

Post by orf »

the_mitsuhiko wrote: post #69
orf wrote: post #56

Earlier quoted context omitted.

> From my experience it’s really hard to make async outperform sync when databases are involved because the async layer adds so much overhead Highly disagree as the database is just another IO connection to a server, which is asyncio bread and butter. Being able to stream data from longer running queries without buffering and whilst serving other requests (and making other queries) is really quite powerful. But yeah,…

The database is mostly just idle IO. You send a query and then you wait for results. That’s something sync python is decent at because when you wait for that IO the GIL is released. The situation is different if there is a lot of activity on the epoll/kqueue etc. (connects, data ready etc.).

Apologies - I completely misread your initial comment. Yeah that's correct.

Despite this I think it's quite rare to hit this limit, at least in the orchestration-style use cases I use asyncio for. With those I value making independent progress on a number of async tasks rather than potentially being blocked waiting for a worker thread to become available.

Re: Async Python is not faster

#77

Post by arethuza »

barrkel wrote: post #73
arethuza wrote: post #68

Earlier quoted context omitted.

In a purely sync world you can have tens of threads per processes without much difficulty - so definitely much heavier resource wise than async but not that bad.

10s? There are over 2000 threads running on my Linux laptop, with a couple of database servers, an IDE and two browsers open. Firefox has 323 threads, Chrome 420, and Slack 69. Async code with continuations and no second execution of continuations is isomorphic to threaded code. The chief differences are that in threaded code, the pointer to the continuation function and its closure are on the stack instead of on the…

Yeah - I knew it was large numbers but didn't have them handy so used 10s :-)

Re: Async Python is not faster

#78

Post by tannhaeuser »

It's about time someone put this into perspective with figures before more and more people rush to implement business apps in async style (= 80's cooperative multiprocessing). There are exceptions of course; for example Node.js was originally envisioned for eg. game servers where async's purported robustness in the presence of a massive number of open sockets supposedly helps. But I think for the vast majority of workloads going async has a terrible impact to your codebase (either with callback hell or by deprecating most of the host language's flow control primitives like try/catch in favour of hard-to-debug ad-hoc constructs such as Promises). Another price to pay is groking Node.js' streams (streams2/streams3) and domain APIs and unhelpful exception handling story with subtle changes even as late as in v13. As I hear, Python's async APIs aren't uncontroversial either.

Now the next thing I'd be interested to get debunked is multithreading vs multiple processes with shared memory (SysV shmem). I'm not very sure, but I'd not been surprised to hear that the predominance of multithreaded runtimes (JVM, most C++ appservers) is purely a cargo-cult effect. As far as I remember, threads were introduced for small and isolated problems in GUI programs, like code completion in IDEs; they were never intended for replacing O/S processes and their isolation guarantees.

Re: Async Python is not faster

#79

Post by wwright »

hombre_fatal wrote: post #74

One big difference between one thread per request vs single-threaded async code is that synchronization and accessing shared resources is trivial when all of your code is running on a single thread. An entire category of data races like `x += 1` become impossible without you even thinking about it. And that's often worth it for something like a game server where everything is beating on the same data structures. I do…

FWIW, Rust gives you the same simplicity (no data races at runtime) with threads as well.

(Not necessarily on topic, but if you’re really excited about dodging data races, I figured it would give you something fun to look at!)

Re: Async Python is not faster

#80

Post by rlpb »

I find it interesting that all the talk here is about performance, and nobody has mentioned any benefits of Async Python when performance isn't an issue.

I use trio/asyncio to more easily write correct complex concurrent code when performance doesn't matter. See "The Problem with Threads"[1].

For this use case, Async Python probably still isn't faster, but that doesn't matter. Let's not throw out the baby with the bathwater :)

[1] https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...

Post reply on HN