Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

191–200 of 364 posts

Re: Async Python is not faster

#191

Post by camgunz »

danbruc wrote: post #135
zzzeek wrote: post #125

Earlier quoted context omitted.

> When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some overhead, but that bit of overhead is probably a lot less than the 3 billion cpu cycles you'll waste waiting 1000ms for an external service. but threads get you th…

but threads get you the same thing with much less overhead. That is not true, at least not in general, the whole point of using continuations for async I/O is to avoid the overhead of using threads, the scheduler overhead, the cost of saving and restoring the processor state when switching tasks, the per thread stack space, and so on.

async implementations build a scheduler into the runtime, and that's generally slower than the OS' scheduler. 10-100x slower if it's not in C (or whatever).

Re: Async Python is not faster

#192

Post by throwaway894345 »

jillesvangurp wrote: post #7
berbc wrote: post #4

Is speed really a good reason for using async? If I remember correctly, asynchronous I/O was introduced to deal with many concurrent clients. Therefore, I would have liked to see how much memory all those workers use, and how many concurrent connections they can handle.

I think speed is the wrong word here. A better word is throughput. The underlying issue with python is that it does not support threading well (due to the global interpreter lock) and mostly handles concurrency by forking processes instead. The traditional way of improving throughput is having more processes, which is expensive (e.g. you need more memory). This is a common pattern with other languages like ruby, php,…

> which is expensive (e.g. you need more memory)

Memory is cheap; the cost is in constant de/serialization. Same with "just rewrite the hotspots in C!"-style advice; de/serialization can easily eat anything you saved by multiprocessing/rewriting. Python is a deceivingly hard language, and a lot of this is a direct result of the "all of CPython is the public C-extension interface!" design decision (significant limitations on optimizations => heavy dependency on C-extensions for anything remotely performance sensitive => package management has to deal extensively with the nightmare that is C packaging => no meaningful cross-platform artifacts or cross compilation => etc).

Re: Async Python is not faster

#194

Post by camgunz »

brightball wrote: post #118
calpaterson wrote: post #105

Earlier quoted context omitted.

I think it is surprising to a lot of people who do take it as read that async will be faster. As I describe in the first line of my article I don't think that people who think async is faster have unreasonable expectations. It seems very intuitive to assume that greater concurrency would mean greater performance - at least one some measure. > When you're dealing with external REST APIs that take multiple seconds to r…

This is often a point of confusion for people when looking at Erlang, Elixir or Go code. Concurrency beyond leveraging available CPU's doesn't really add any advantage. On the web when the bulk of your application code time is waiting on APIs, database queries, external caches or disk I/O it creates a dramatic increase in the capacity of your server if you can do it with minimal RAM overhead. It's one of the big reas…

> On the web when the bulk of your application code time is waiting on APIs, database queries, external caches or disk I/O it creates a dramatic increase in the capacity of your server if you can do it with minimal RAM overhead.

Python doesn't block on I/O.

Re: Async Python is not faster

#195

Post by catalogia »

crimsonalucard1 wrote: post #154
phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

I see this elitist attitude all over the internet. First it was people saying “Guys why are you over reacting to corona the flu is worse.” Then it was people saying “Guys, stop buying surgical masks, The science says they don’t work it’s like putting a rag over your mouth.” All of these so called expert know it alls were wrong and now we have another expert on asynchronous python telling us he knows better and he’s n…

You're making the classic mistake of assuming a common thread connects the people who've annoyed you in various unrelated contexts.

Re: Async Python is not faster

#196

Post by szatkus »

zzzeek wrote: post #153
szatkus wrote: post #151

Earlier quoted context omitted.

CPU can do billions of operations every second. When you have 200ms for every request that overhead is not that large, you're still blocked by I/O.

for local services like databases, real world benchmarks disagree.

You should add that you mean just databases. I've just looked at your profile and as I understand it's your focus.

I built a service that was making a lot of requests. Much enough that at some point we've run out of 65k connections limit for basic Linux polling (we needed to switch to kpoll). Some time after that we've ran out of other resources and switching from threads to threads+greenlets really solved our problem.

Re: Async Python is not faster

#197

Post by ben509 »

crimsonalucard1 wrote: post #154
phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

I see this elitist attitude all over the internet. First it was people saying “Guys why are you over reacting to corona the flu is worse.” Then it was people saying “Guys, stop buying surgical masks, The science says they don’t work it’s like putting a rag over your mouth.” All of these so called expert know it alls were wrong and now we have another expert on asynchronous python telling us he knows better and he’s n…

> Nodejs blew everything out of the water

Node's JIT comes from a web browser's javascript implementation used by billions of people. It's also had async baked in from day one.

Python started single process, added threading, and then bolted async on top of that. And CPython is a pretty straight interpreter.

A comparison between Node and PyPy would be more informative, but PyPy has a far less mature JIT and still has to deal with Python's dynamism.

> If we invent a label maybe they’ll start becoming self aware and start acting more down to earth.

You can't lecture people into self-awareness, any more than experts can lecture everyone into wearing masks.

Re: Async Python is not faster

#198

Post by hypewatch »

Why do you use less than half the workers for the async libraries?

uwsggi+flask - 16 workers unicorn+starlette - 5 workers

The highest throughout examples in your benchmark all have 16 workers. I also don’t see any hardware data... Does your machine have 16 cores?

Re: Async Python is not faster

#199

Post by Jasper_ »

ben509 wrote: post #186
Jasper_ wrote: post #152

Earlier quoted context omitted.

People seem to keep misunderstanding the GIL. It's the Global Interpeter Lock, and it's effectively the lock around all Python objects and structures. This is necessary because Python objects have no thread ownership model, and the development team does not want per-object locks. During any operation that does not need to modify Python objects, it is safe to unlock the GIL. Yielding control to the OS to wait on I/O i…

To clarify that the CPython devs aren't being arbitrary here: There have been attempts at per-object or other fine-grained locking, and they appear to be less performant than a GIL, particularly for the single-threaded case. Single-threaded performance is a major issue as that's most Python code.

Yes. I expect generic fine-grained locking, especially per-object leaks, to be less performant for multi-threaded code too, as locks aren't cheap, and even with the GIL, lock overhead could still be worse than a good scheduler.

Any solution which wants to consider per-object locking has to consider removing refcounting, or locking the refcount bits separately, as locking/unlocking objects to twiddle their refcounts is going to be ridiculously expensive.

Ultimately, the Python ownership and object model is not condusive to proper threading, as most objects are global state and can be mutated by any thread.

Re: Async Python is not faster

#200

Post by calpaterson »

hypewatch wrote: post #198

Why do you use less than half the workers for the async libraries? uwsggi+flask - 16 workers unicorn+starlette - 5 workers The highest throughout examples in your benchmark all have 16 workers. I also don’t see any hardware data... Does your machine have 16 cores?

Hi - this is explained in detail in the article.
Post reply on HN