Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

231–240 of 364 posts

Re: Async Python is not faster

#231

Post by Jtsummers »

rukittenme wrote: post #108
rlpb wrote: post #80

I find it interesting that all the talk here is about performance, and nobody has mentioned any benefits of Async Python when performance isn't an issue. I use trio/asyncio to more easily write correct complex concurrent code when performance doesn't matter. See "The Problem with Threads"[1]. For this use case, Async Python probably still isn't faster, but that doesn't matter. Let's not throw out the baby with the ba…

Whats the point of writing concurrent code if its not faster?

Contrasting with jdlshore, concurrency can make programs much easier to reason about, when done well. This is a benefit of both Go and Erlang, though they use different approaches.

Concurrency can help you separate out logic that is often commingled in non-concurrent code, but doesn't need to be. As a real-world example, I used to do safety critical systems for aircraft. The linear, non-concurrent version, included a main loop that basically executed a couple dozen functions. Each function may or may not have dependencies on the other functions, so information was passed between them over multiple passes through this main loop (as their order was fixed) using shared memory.

A similar project had about a dozen processes, each running concurrently. There was no speed improvement, but the connection between each activity was handled via channels (equivalent in theory to Go's channels, less like Erlang's mailboxes as the channels could be shared). We knew it was correct because each process was a simple state machine, separated cleanly from all other state machines.

The second system's code was much simpler, there was no juggling (in our code) of the state of the system, compared to managing the non-concurrent logic. If a channel had data to be acted on, the process continued, otherwise it waited. Very simple. And it turns out that many systems can be modeled in a similar fashion (IME). Of course, we had a very straightforward communication mechanism (again, essentially the same as Go channels except it was a library written in, as I recall, Ada by whoever made the host OS).

Re: Async Python is not faster

#232

Post by didibus »

mavdi wrote: post #206
phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

I get enraged when articles like this get upvotes. The evidence given doesn't at all negate the reasoning behind using async, which as you said, is about not having to be blocked by IO, not freaking throughput test for an unrealistic scenario. Just goes to show the complete lack of understanding of the topic. I wouldn't dare write something up if I didn't 100% grasp it, but the bar is way lower for some others it see…

I don't know the async Python specifics, but from what I understand, you don't necessarily need async to handle large number of IO requests, you can simply use non-blocking IO and check back on it synchronously either in some loop or at specific places in your program.

The use of async either as callbacks, or user threads, or coroutines, is a convenience layer for structuring your code. As I understand, that layer does add some overhead, because it captures an environment, and has to later restore it.

Re: Async Python is not faster

#233

Post by zzzeek »

ashtonkem wrote: post #174
phodge wrote: post #93

How is this result surprising? The point of coroutines isn't to make your code execute faster, it's to prevent your process sitting idle while it waits for I/O. When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some over…

We need to stop saying “faster” with regards to async. The point of async was always either fitting more requests per compute resource, and/or making systems more latency consistent under load. “Faster” is misleading because the speed improvements that you get with async is very dependent on load. At low levels there is going to typically be negligible or no speed gains, but at higher levels the benefit will be incre…

you just contradicted yourself:

> “Faster” is misleading

and

> "At low levels there is going to typically be negligible or no speed gains, but at higher levels the benefit will be incredibly obvious."

there are no "speed" gains period. the same amount of work will be accomplished in the same amount of time with threads or async. async makes it more memory efficient to have a huge number of clients waiting concurrently for results on slow services, but all of those clients walking off with their data will not be reached "faster" than with threads.

the reason that asyncio advocates say that asyncio is "faster" is based on the notion that the OS thread scheduler is slow, and that async context switches are some combination of less frequent and more efficient such that async is faster. This may be the case for other languages but for Python's async implementations it is not the case, and benchmarks continue to show this.

Re: Async Python is not faster

#234

Post by ashtonkem »

zzzeek wrote: post #233
ashtonkem wrote: post #174

Earlier quoted context omitted.

We need to stop saying “faster” with regards to async. The point of async was always either fitting more requests per compute resource, and/or making systems more latency consistent under load. “Faster” is misleading because the speed improvements that you get with async is very dependent on load. At low levels there is going to typically be negligible or no speed gains, but at higher levels the benefit will be incre…

you just contradicted yourself: > “Faster” is misleading and > "At low levels there is going to typically be negligible or no speed gains, but at higher levels the benefit will be incredibly obvious." there are no "speed" gains period. the same amount of work will be accomplished in the same amount of time with threads or async. async makes it more memory efficient to have a huge number of clients waiting concurrentl…

I did not contradict myself; saying that async is “faster” implies speed gains in all circumstances. In reality the benefits of async io is extremely load dependent, which is why I don’t want to call it “faster”.

Re: Async Python is not faster

#235

Post by WoodenChair »

1337shadow wrote: post #137
WoodenChair wrote: post #85

Earlier quoted context omitted.

You perfectly illustrated why this is a problem. Calling functions from one side to the other involves ceremony. Ceremony adds cognitive overhead and decreases readability.

Would that work for you ? result = asyncio.run(red(x)) That's calling async function red from non-async code. Not seeing any particular readability issue with that usage neither. If you don't call asyncio.run or await on the result of an async function call, then you get a coroutine for result.

> result = asyncio.run(red(x))

Still much harder to think about/read then

   result = red(x)
We should be finding ways to get to the latter with concurrency. async/await is at best a patchwork compromise until we can do better.

Re: Async Python is not faster

#236

Post by Grimm1 »

meritt wrote: post #219
Grimm1 wrote: post #150

Async is useful for high IO where you may have a lot of down time between the requests. Are you pulling many requests from different servers with different response times, communicating with a db or pulling out large response bodies. Async is probably going to do better since each one of those synchronously represents potentially large idling periods where other requests could have gotten work done. As to the article…

> forking 16 instances is going to be a lot heavier on memory It really depends on how the application is designed. Fork operates through mmap and copy-on-write. It's extremely lightweight by default. A well-designed fork-based application will already have everything necessary to run a given process into memory, not munge any of the existing shared memory, and only allocate and free memory associated with new events…

"All the workers are sharing the exact same core application code and logic in memory."

Oh interesting, are you saying an intelligent forking implementation is able to share static portions of memory with multiple children?

I was perhaps under the naive assumption forking was pretty much just a full memory copy of the parent.

Re: Async Python is not faster

#237

Post by kirkeby »

camgunz wrote: post #194
brightball wrote: post #118

Earlier quoted context omitted.

This is often a point of confusion for people when looking at Erlang, Elixir or Go code. Concurrency beyond leveraging available CPU's doesn't really add any advantage. On the web when the bulk of your application code time is waiting on APIs, database queries, external caches or disk I/O it creates a dramatic increase in the capacity of your server if you can do it with minimal RAM overhead. It's one of the big reas…

> On the web when the bulk of your application code time is waiting on APIs, database queries, external caches or disk I/O it creates a dramatic increase in the capacity of your server if you can do it with minimal RAM overhead. Python doesn't block on I/O.

Of course it does.

Re: Async Python is not faster

#238

Post by dragonwriter »

otabdeveloper4 wrote: post #168
willseth wrote: post #161

Earlier quoted context omitted.

> But blocking IO isn't one of those situations, so you can just use threads. Threads and async are not mutually exclusive. If your system resources aren't heavily loaded, it doesn't matter, just choose the library you find most appropriate. But threads require more system overhead, and eventually adding more threads will reduce performance. So if it's critical to thoroughly maximize system resources, and your system…

> But threads require more system overhead, and eventually adding more threads will reduce performance. Absolutely false. OS threads are orders of magnitude lighter than any Python coroutine implementation.

> OS threads are orders of magnitude lighter than any Python coroutine implementation.

But python threads, which have extra weight on top of an cross-platform abstraction layer on top of the underlying OS threads, are not lighter than python coroutines.

You aren't choosing between Python threads and unadorned OS threads when writing Python code.

Re: Async Python is not faster

#239

Post by throwaway894345 »

Grimm1 wrote: post #224
throwaway894345 wrote: post #221

Earlier quoted context omitted.

Double-digit interpreters per host? Where is the expense? Interpreters have a relatively small memory overhead (<10mb). If you're running 100 interpreters per host (you shouldn't be), that's an extra $50/host/year. But you should be running <10/host, so an extra $5/host/year. Not ideal, but not "expensive", and if you care about costs your biggest mistake was using Python in the first place.

I don't know where you're seeing the Another google search shows me Gunicorn, for instance, using high memory on fork isn't exactly uncommon either. Edit: I reworded some stuff up there and tried to make my point more clear.

The interpreter overhead on macos is 7.7mb. I can't speak to gunicorn configuration but it's far from the only game in town.

Re: Async Python is not faster

#240

Post by dullgiulio »

tijsvd wrote: post #155
yamrzou wrote: post #123

Earlier quoted context omitted.

> is not necessarily a performance improvement over a sync version that just starts more workers and has the OS kernel scheduler organise things Co-routines are not necessarily faster than threads, but they yield to a performance improvement if one has to spin thousands of them : they have less creation overhead and consume less RAM.

> Co-routines are not necessarily faster than threads, but they yield to a performance improvement if one has to spin thousands of them : they have less creation overhead and consume less RAM. This hardly matters when spinning up a few thousand threads. Only memory that's actually used is committed, one 4k page at a time. What is 10MB these days? And that is main memory, while it's much more interesting what fits in…

Languages that to green threads don't do them for memory savings, but to save on context switches when a thread is blocked and cannot run. System threads are scheduled by the OS, green threads my the language runtime, which saves a context switch.
Post reply on HN