Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The magic of asyncio explained

hackernoon.com

Post reply on HN
Print view View original

11–20 of 95 posts

Re: The magic of asyncio explained

#11

Post by kjeetgill »

sadgit wrote: post #3

I’m still bummed that Python took this direction. Maybe introducing new keywords into the language for event loop concurrency was Python’s way of satisfying “explicit is better than implicit” but i can’s shake the feeling that callback passing and generator coroutines are a fad that is complex enough to occupy the imagination of a generation of programmers while offering little benefit compared to green threads.

I agree it's always left a bag taste in my mouth. I loved generators and yield/yield from, but I was stuck on 2.7 for a long time so I never quite understood the motivation for async/await over them.

One issue is that it "reifys" the "colored function" problem that green threads like goroutines don't have!

Side note: Java world is working on green threads/fibers for the JVM in Project Loom.

[0]: http://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.htm...

Re: The magic of asyncio explained

#12

Post by toast0 »

kbaker wrote: post #8
shawn wrote: post #6

Earlier quoted context omitted.

Effing thank you. I don't think most people realize just how convenient green threads are. It kills me to see devs stuck in the local maximum of callback hell. That sad, https://www.usenix.org/system/files/conference/atc12/atc12-f... raises some interesting points in defense of one-way RPC. The key is not to allow returns.

Isn't Python's async/await syntax an implementation of green threads? I mean using await is almost exactly the cooperative scheduling idea. The article may use Futures and callbacks but you can just as easily do something like: result = await fake_network_request('one')

They're sort of similar, and you can probably get the same work done in either system, but I think real threading (green or otherwise), may leave you with less cognitive load. Spawning a thread may be complex, and thinking about how the threads are scheduled is often complex, but what each thread does can be very simple -- and you don't have to think about 'long running things need to be futured/awaited', you just do things in a straightforward way in the thread (caveat: slightly less straightforward if you need thread actions to be cancellable).

Green threads may be running an event loop underneath, but it's a useful abstraction in many contexts.

Re: The magic of asyncio explained

#13

Post by cshenton »

sadgit wrote: post #3

I’m still bummed that Python took this direction. Maybe introducing new keywords into the language for event loop concurrency was Python’s way of satisfying “explicit is better than implicit” but i can’s shake the feeling that callback passing and generator coroutines are a fad that is complex enough to occupy the imagination of a generation of programmers while offering little benefit compared to green threads.

For me it had the opposite effect.

Working with async-await syntax was the last straw that made me finally go "there's got to be a better way" and find a language can handle concurrency without the semantic overhead (in my case Go, but there are others).

Re: The magic of asyncio explained

#14

Post by yen223 »

toast0 wrote: post #12
kbaker wrote: post #8

Earlier quoted context omitted.

Isn't Python's async/await syntax an implementation of green threads? I mean using await is almost exactly the cooperative scheduling idea. The article may use Futures and callbacks but you can just as easily do something like: result = await fake_network_request('one')

They're sort of similar, and you can probably get the same work done in either system, but I think real threading (green or otherwise), may leave you with less cognitive load. Spawning a thread may be complex, and thinking about how the threads are scheduled is often complex, but what each thread does can be very simple -- and you don't have to think about 'long running things need to be futured/awaited', you just do…

Python has proper threads, and they're anything but simple.

Re: The magic of asyncio explained

#15

Post by nerdwaller »

sadgit wrote: post #3

I’m still bummed that Python took this direction. Maybe introducing new keywords into the language for event loop concurrency was Python’s way of satisfying “explicit is better than implicit” but i can’s shake the feeling that callback passing and generator coroutines are a fad that is complex enough to occupy the imagination of a generation of programmers while offering little benefit compared to green threads.

Having worked in asyncio for a bit I don’t entirely follow this (it truly could just be familiarity), very little of asyncio (especially post `async/await` were introduced in the language) is callback based and reads more procedural.

Regarding generator coroutines it feels like a natural evolution of the language. Given that yield previously suspended the current function’s state providing value(s) to the closure, it only makes sense that yield (on the producer side)/await (on the consumer side) does the same thing but in an event loop based context.

I can’t speak deeply enough about green threads, but from my understanding there’s much less magic (as you cite “explicit”) in an async/await world vs the magic (“implicit”) world of green threads.

    async def thing():
        print(‘before’)
        await asyncio.sleep(0)
        print(‘after’)
Vs

    def thing():
        print(‘before’)
        gevent.sleep(0)
        print(‘after’)
There’s nothing clear in the latter when something yields or otherwise passes control flow.

Having worked in a few evented systems, I find the explicit shift to the runtime is valuable.

Re: The magic of asyncio explained

#16

Post by pdkl95 »

toast0 wrote: post #12
kbaker wrote: post #8

Earlier quoted context omitted.

Isn't Python's async/await syntax an implementation of green threads? I mean using await is almost exactly the cooperative scheduling idea. The article may use Futures and callbacks but you can just as easily do something like: result = await fake_network_request('one')

They're sort of similar, and you can probably get the same work done in either system, but I think real threading (green or otherwise), may leave you with less cognitive load. Spawning a thread may be complex, and thinking about how the threads are scheduled is often complex, but what each thread does can be very simple -- and you don't have to think about 'long running things need to be futured/awaited', you just do…

> ... you don't have to think about 'long running things need to be futured/awaited', you just do things in a straightforward way in the thread

https://www.youtube.com/watch?v=bzkRVzciAZg

Six years later, very little has change in the arguments about events vs threads.

Re: The magic of asyncio explained

#17

Post by sebcat »

sadgit wrote: post #3

I’m still bummed that Python took this direction. Maybe introducing new keywords into the language for event loop concurrency was Python’s way of satisfying “explicit is better than implicit” but i can’s shake the feeling that callback passing and generator coroutines are a fad that is complex enough to occupy the imagination of a generation of programmers while offering little benefit compared to green threads.

> i can’s shake the feeling that callback passing and generator coroutines are a fad [...]

Callback passing and coroutines are well known techniques that's been around for a while. Generators are just coroutines that yield to their parent. I remember using these concepts in C and Tcl ~15 years ago[1] and they were well known then. According to wikipedia (citing Knuth), the term coroutine was coined in the late 50's by Conway.

Callback passing and coroutines suits some problems well. Sure there are situations where they do not fit and if people use a hammer for all of their problems they will create new ones. I wouldn't call it a fad, though the techniques may be a bit hyped up in some circles.

[1] I don't remember when Tcl got the coroutine package, that may have been later

Re: The magic of asyncio explained

#18

Post by greyman »

Python is my language of first choice, but I must say that I am not that thrilled how this multithreading ended up. There are many tutorials about the topic promising to explain how it works, usually in the form of "simple introduction". But when one tries to implement something production-ready, with correct error handling etc., things starts to complicate pretty quickly; at least that was my experience. I don't want to accuse anyone specifically, but most of the tutorials I saw seems to portrait it in a way, that it looks easier than it actually is.

Ultimately, my company decided, that instead of fighting with asyncio, certain projects will switch to Go.

Re: The magic of asyncio explained

#19

Post by _Codemonkeyism »

Wasn't there a language where every call was async? Instead of async ... A/returing Future[A] it did/would return A from method calls.

If it didn't exist, one can imagine one.

A.x = 3 would be wrapped in A.map(_.x = 3) etc. So you write code that would be executed when you finally await a value. No more red/blue world. Would probably need coroutines instead of threads for executing.

Post reply on HN