Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python and Async Simplified (2018)

aeracode.org

Post reply on HN
Print view View original

11–20 of 47 posts

Re: Python and Async Simplified (2018)

#11

Post by BiteCode_dev »

uniqueuid wrote: post #7
BiteCode_dev wrote: post #5

It gives good pointers but it falls short on the usual suspects for an article on asyncio. When teaching it, it's important to emphasis: - await is locally blocking, so you should isolate linear workflows into their own coro, which is the unit of concurrency. - to allow concurrency, you should use asyncio.create_task on coro (formerly ensure_future). - you should always explicitly delimitate the life cycle of any tas…

> you should always explicitly delimitate the life cycle of any task Unless you want a hacky actor system, in which case it's totally fine to `create_task` a ton of corountines which have their own spin loop with await sleep :)

At some point even those tasks must be cleanly stopped and unless you want to play erlang and "let it crash", the actors have a lifecycle as well. Making it explicit will avoid much pain, and ease testing a lot. Also it will make resources consumption more predictable.

Re: Python and Async Simplified (2018)

#12

Post by ogogmad »

I thought Python couldn't multithread because of GIL? I understood from the article that async derives all its benefit from certain OS-level operations which don't need to run in a coroutine, like reading from a network socket or waiting for timers to finish.

Another question: Is Python's implementation of async/await identical to other languages? In particular, do they always use coroutines instead of threads?

Re: Python and Async Simplified (2018)

#13

Post by ogogmad »

landersg wrote: post #8

Async is overengineered and bolted on. If you must use Python, I'd still recommend Twisted, which is more accessible. Otherwise, of course use Go, Elixir, etc. in the first place.

I don't know. It seems to me Elixir requires a new and rather restrictive programming paradigm. For me, that's even worse than being overengineered.

Re: Python and Async Simplified (2018)

#14

Post by diarrhea »

ogogmad wrote: post #12

I thought Python couldn't multithread because of GIL? I understood from the article that async derives all its benefit from certain OS-level operations which don't need to run in a coroutine, like reading from a network socket or waiting for timers to finish. Another question: Is Python's implementation of async/await identical to other languages? In particular, do they always use coroutines instead of threads?

I understand async/await in Python to be entirely single-threaded. So is, for example, C#'s implementation: https://learn.microsoft.com/en-us/dotnet/csharp/programming-... ("The async and await keywords don't cause additional threads to be created.")

Re: Python and Async Simplified (2018)

#15

Post by aserafini »

uniqueuid wrote: post #7
BiteCode_dev wrote: post #5

It gives good pointers but it falls short on the usual suspects for an article on asyncio. When teaching it, it's important to emphasis: - await is locally blocking, so you should isolate linear workflows into their own coro, which is the unit of concurrency. - to allow concurrency, you should use asyncio.create_task on coro (formerly ensure_future). - you should always explicitly delimitate the life cycle of any tas…

> you should always explicitly delimitate the life cycle of any task Unless you want a hacky actor system, in which case it's totally fine to `create_task` a ton of corountines which have their own spin loop with await sleep :)

Even if you want to ‘fire and forget’, it’s still essential to keep a reference to the task, otherwise it can be garbage collected mid-execution:

https://docs.python.org/3/library/asyncio-task.html#asyncio....

Re: Python and Async Simplified (2018)

#16

Post by bsenftner »

landersg wrote: post #8

Async is overengineered and bolted on. If you must use Python, I'd still recommend Twisted, which is more accessible. Otherwise, of course use Go, Elixir, etc. in the first place.

It reminds me of the cooperative multitasking used by the original Mac OS.

Re: Python and Async Simplified (2018)

#17

Post by throwamon »

ogogmad wrote: post #13
landersg wrote: post #8

Async is overengineered and bolted on. If you must use Python, I'd still recommend Twisted, which is more accessible. Otherwise, of course use Go, Elixir, etc. in the first place.

I don't know. It seems to me Elixir requires a new and rather restrictive programming paradigm. For me, that's even worse than being overengineered.

> new

Erlang has been around since what, the 80s? Elixir is "just" Erlang with a different face and extra features.

> restrictive

which is? Functional programming? Immutability?

Interestingly, Erlang is often called a "true" object-oriented language thanks to its actor model. It's incredibly powerful and flexible, pretty much the opposite of restrictive. Just for a simple example, you can inspect, debug and modify your program while it's running.

From your comment it just seems you're not familiar with it.

Re: Python and Async Simplified (2018)

#18

Post by throwamon »

landersg wrote: post #8

Async is overengineered and bolted on. If you must use Python, I'd still recommend Twisted, which is more accessible. Otherwise, of course use Go, Elixir, etc. in the first place.

Promising contenders include algebraic effects in OCaml and JVM's Project Loom.

Re: Python and Async Simplified (2018)

#19

Post by quietbritishjim »

BiteCode_dev wrote: post #5

It gives good pointers but it falls short on the usual suspects for an article on asyncio. When teaching it, it's important to emphasis: - await is locally blocking, so you should isolate linear workflows into their own coro, which is the unit of concurrency. - to allow concurrency, you should use asyncio.create_task on coro (formerly ensure_future). - you should always explicitly delimitate the life cycle of any tas…

> ... TaskGroup will help when it becomes mainstream.

Strongly agreed, but you can use anyio [1] in to of asyncio to get that functionality right now. Or, maybe even better, use Trio [2] instead, which is where the idea came from in the first place.

[1] https://anyio.readthedocs.io/en/stable/

[2] https://trio.readthedocs.io/en/stable/

Re: Python and Async Simplified (2018)

#20

Post by zzzeek »

ogogmad wrote: post #12

I thought Python couldn't multithread because of GIL? I understood from the article that async derives all its benefit from certain OS-level operations which don't need to run in a coroutine, like reading from a network socket or waiting for timers to finish. Another question: Is Python's implementation of async/await identical to other languages? In particular, do they always use coroutines instead of threads?

since it's my job to clear these things up, a few pointers:

1. python has threads. they just cannot perform CPU bound tasks in parallel due to the GIL. The GIL is released for IO, so threads can perform IO waiting in parallel, just like asyncio 2. asyncio runs in one thread, and has the exact same limitations as threads as implemented in Python, CPU operations are serialized, async tasks can yield for IO.

the advantages offered by asyncio are: 1. you can have thousands of tasks extremely quickly cheaply, which is not as much the case for threads in Python . this can allow for massive concurrent architectures more expediently, provided your concurrency is very IO bound (if you are CPU bound, disaster) 2. people just like asyncio's programming model, IMPO this is largely due to the popularity of Javascript's event-based model being natural for lots of newer programmers

Post reply on HN