Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python is not faster

calpaterson.com

Post reply on HN
Print view View original

261–270 of 364 posts

Re: Async Python is not faster

#261

Post by tijsvd »

dullgiulio wrote: post #240
tijsvd wrote: post #155

Earlier quoted context omitted.

> Co-routines are not necessarily faster than threads, but they yield to a performance improvement if one has to spin thousands of them : they have less creation overhead and consume less RAM. This hardly matters when spinning up a few thousand threads. Only memory that's actually used is committed, one 4k page at a time. What is 10MB these days? And that is main memory, while it's much more interesting what fits in…

Languages that to green threads don't do them for memory savings, but to save on context switches when a thread is blocked and cannot run. System threads are scheduled by the OS, green threads my the language runtime, which saves a context switch.

Green threads are scheduled by the language runtime and by the OS. If the OS switches from one thread to another in the same process, there is no context switch, really, apart from the syscall itself which was happening anyway (the recv that blocks and causes the switch). At least not on Linux, where I've measured the difference.

Re: Async Python is not faster

#262

Post by gotzmann »

Thats interesting. In PHP world the situation radically different: modern frameworks based on libevent are really speed up web apps up to 10x.

I've thoroughly benchmarked my own framework[1] for REST APIs and now it outperforms many of Go / Node.js platforms on Techempower[2]

[1] https://github.com/gotzmann/comet

[2] https://www.techempower.com/benchmarks/#section=test&runid=e...

Re: Async Python is not faster

#263

Post by jdlshore »

rukittenme wrote: post #255
Jtsummers wrote: post #231

Earlier quoted context omitted.

Contrasting with jdlshore, concurrency can make programs much easier to reason about, when done well. This is a benefit of both Go and Erlang, though they use different approaches. Concurrency can help you separate out logic that is often commingled in non-concurrent code, but doesn't need to be. As a real-world example, I used to do safety critical systems for aircraft. The linear, non-concurrent version, included a…

Signals are not dependent on concurrency. And you don't need multiple processes to implement a state machine. I mean think about it. Whats the difference between sending message A and then message B versus sending messages A and B into a queue and letting some async process pop from it? Less complexity and guaranteed message delivery come for free in single-threaded code. Am I wrong? What am I missing?

I don't think you're wrong, but in Jtsummers' specific case, I think multi-processing probably would be simpler. You don't have to implement the event loop, there's no risk of tromping on other processes' data, and if a process gets into an invalid state, you can just die without impacting others.

You'd need a good watchdog and error handling, but presumably some of that came for "free" in their environment.

Although if you take out the "free" OS support, watchdog, etc., I agree that there's likely a place between "shared memory spaghetti" and "multi-processing" that's simpler than both.

Re: Async Python is not faster

#264

Post by detaro »

rukittenme wrote: post #260
detaro wrote: post #259

Earlier quoted context omitted.

"some async process" is a concurrency mechanism, is it not?

It is. The single-threaded example comes before the "versus". The async example comes after. I should have been more clear.

Ah, indeed misread that. Then my answer is: Singlethreaded code sometimes has to implement things an async environment would handle for you.

I.e. when handling many in- and outputs I can write my own loop around epoll etc, write logic to keep of track of queues of data to send per-target etc. Or I can use a runtime that provides that for me and lets me mostly pretend things are running on their own.

Re: Async Python is not faster

#265

Post by catblast »

danbruc wrote: post #135
zzzeek wrote: post #125

Earlier quoted context omitted.

> When you're dealing with external REST APIs that take multiple seconds to respond, then the async version is substantially "faster" because your process can get some other useful work done while it's waiting. Obviously the async framework introduces some overhead, but that bit of overhead is probably a lot less than the 3 billion cpu cycles you'll waste waiting 1000ms for an external service. but threads get you th…

but threads get you the same thing with much less overhead. That is not true, at least not in general, the whole point of using continuations for async I/O is to avoid the overhead of using threads, the scheduler overhead, the cost of saving and restoring the processor state when switching tasks, the per thread stack space, and so on.

The scheduler overhead and the cost of context-switches are vastly overstated compared to alternatives. The per thread stack space in effect has virtually no run-time cost, and starting off at a single 4k page for a stack, thousands still only waste a miniscule about of memory.

Re: Async Python is not faster

#267

Post by meritt »

Grimm1 wrote: post #236
meritt wrote: post #219

Earlier quoted context omitted.

> forking 16 instances is going to be a lot heavier on memory It really depends on how the application is designed. Fork operates through mmap and copy-on-write. It's extremely lightweight by default. A well-designed fork-based application will already have everything necessary to run a given process into memory, not munge any of the existing shared memory, and only allocate and free memory associated with new events…

"All the workers are sharing the exact same core application code and logic in memory." Oh interesting, are you saying an intelligent forking implementation is able to share static portions of memory with multiple children? I was perhaps under the naive assumption forking was pretty much just a full memory copy of the parent.

Yep, but that's simply how the linux kernel works [1]. As a programmer you need to essentially load up all your modules/libraries/data/etc that you will need into memory before the fork, and treat the forked() processes as read-only as much as possible from a resource perspective. If you modify anything from the parent, you get your own page as soon as that happens. [2][3]

[1] https://www.informit.com/articles/article.aspx?p=368650

[2] https://en.wikipedia.org/wiki/Copy-on-write

[3] https://en.wikipedia.org/wiki/Fork_(system_call)

Re: Async Python is not faster

#269

Post by ahupp »

This is true as far as it goes, but is not testing the (very common) areas where async shines.

Imagine you're loading a profile page on some social networking site. You fetch the user's basic info, and then the information for N photos, and then from each photo the top 2 comments, and for each comment the profile pic of the commentor. You can't just fetch all this in one shot because there's data dependencies. So you start fetching with blocking IO, but that makes your wait time for this request proportional to the number of fetches, which might be large.

So instead, you ideally want your wait to be proportional to the depth of your dependency tree. But composing all these fetches that way is hard without the right abstraction. You can cobble it together with callbacks but it gets hairy fast.

So (outside of extreme scenarios) it's not really about whether async is abstractly faster than sync. It's about how real developers would solve the same problem with/without async.

(Source: I worked on product infrastructure in this area for many years at FB)

Re: Async Python is not faster

#270

Post by antoncohen »

wetmore wrote: post #204
antoncohen wrote: post #167

I think this blog post on Python, Gunicorn, and Gevent is relevant: https://rachelbythebay.com/w/2020/03/07/costly/

This blog post is mentioned in TFA

Yes, sorry I should have been more explicit. The article does mention the Rachel by the Bay blog post, but in a way that makes it sound like a different issue. I think they are more related.

I think the Rachel by the Bay blog post does a good job of explaining what event loops are doing under the hood, and how that can lead to bad tail latency for web requests.

Post reply on HN