Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Waiting in Asyncio

hynek.me

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Waiting in Asyncio

#21

Post by nall10 »

hynek wrote: post #17
dilandau wrote: post #2

All these subtleties and gotchas signify, to me, a shit implementation and a fragmented interface. The exact opposite of the Zen of python.

A more charitable view is that they signify an API that evolved from callback-based APIs like Twisted over many years and made nicer APIs like curio or trio only possible (whose features/insights slowly feed back into asyncio but backward compat keeps the warts around).

Evolved? Twisted has a spectacular API, the only thing that is lacking is documentation.

But even without docs I understood it faster than asyncio and it is more pleasant to use.

Twisted is very elegant, but somehow elegant things get eradicated in the Python world. Time and again.

Re: Waiting in Asyncio

#22

Post by t-writescode »

There were 2 things about asyncio that I really didn't like:

1. the library constantly changed under our feet and that also meant that Stack Overflow articles were all over the place between Python 3.5 and 3.7.

2. I don't think I've ever successfully mocked an async function in python. This made testing async code in python, especially mocking responses from an async web call (an obvious use of async in python) very difficult, if memory serves me. It's been several months since I've used async in python (job change) though, so maybe it's gotten easier or maybe the exact mocking test case issues I had are different drom what I think.

Re: Waiting in Asyncio

#24

Post by xtreak29 »

t-writescode wrote: post #22

There were 2 things about asyncio that I really didn't like: 1. the library constantly changed under our feet and that also meant that Stack Overflow articles were all over the place between Python 3.5 and 3.7. 2. I don't think I've ever successfully mocked an async function in python. This made testing async code in python, especially mocking responses from an async web call (an obvious use of async in python) very…

From Python 3.8 there is AsyncMock : https://docs.python.org/3/library/unittest.mock.html#unittes... . It's also available on mock backport from pypi for other versions. It's based on asynctest module. https://pypi.org/project/asynctest/

Re: Waiting in Asyncio

#25

Post by nerdwaller »

j88439h84 wrote: post #14
nerdwaller wrote: post #12

Earlier quoted context omitted.

To save others the search regarding a nursery: https://trio.readthedocs.io/en/stable/tutorial.html#okay-let... I don’t see much more useful here than understanding the asyncio primitives and available synchronization abstractions, just a different API mostly overlapping the same need To be clear, trio also has more than one way to wait for a task - any await is the same (e.g. `await trio.sleep(N)` (e.g. `await asynci…

I'm on mobile so cant type a long response but IME its quite different in terms of ease of use, complexity, and correctness. For a theoretical take see https://vorpus.org/blog/notes-on-structured-concurrency-or-g... To be precise, to wait for a task its actually just await. Theres no ensure_future etc. Trio doesn't have the concept of futures or promises at all, and await f() is treated as a single piece of syntax, s…

Fair enough, the ensure_future() vs create_task() has confused me more than once. It would have been nice if there was a different namespace for the lower level API. Even so, the docs aren’t even clear on when which of the two is appropriate: https://docs.python.org/3/library/asyncio-task.html#asyncio..... https://docs.python.org/3/library/asyncio-future.html#asynci....

Re: Waiting in Asyncio

#26

Post by naasking »

j88439h84 wrote: post #14
nerdwaller wrote: post #12

Earlier quoted context omitted.

To save others the search regarding a nursery: https://trio.readthedocs.io/en/stable/tutorial.html#okay-let... I don’t see much more useful here than understanding the asyncio primitives and available synchronization abstractions, just a different API mostly overlapping the same need To be clear, trio also has more than one way to wait for a task - any await is the same (e.g. `await trio.sleep(N)` (e.g. `await asynci…

I'm on mobile so cant type a long response but IME its quite different in terms of ease of use, complexity, and correctness. For a theoretical take see https://vorpus.org/blog/notes-on-structured-concurrency-or-g... To be precise, to wait for a task its actually just await. Theres no ensure_future etc. Trio doesn't have the concept of futures or promises at all, and await f() is treated as a single piece of syntax, s…

An interesting idea, essentially Trio restructures concurrent code to more closely resemble parallel code, which innately has the property that the concurrency is not observable, ie. the black box property. It's a little more general than strict parallel code though because task spawning is reified as a first class value via which the program can spawn new tasks.

I'm not sure it's totally novel though. For instance, C# has AsParallel() extensions which let you run collection operations in parallel (similar functions in Haskell too). It has the same black box behaviour described by the article, and there's an equivalence between direct control flow in code and indirect control flow reified as a data structure, ie. Haskell's case that lazy evaluation and lists are the only control flow construct you need.

Still, it's an interesting imperative incarnation of the idea!

Re: Waiting in Asyncio

#27

Post by ghostwriter »

toolslive wrote: post #18

- wouldn't it be nice if you could figure out if a function/method could block? - What's a good strategy to migrate a larger python codebase towards asyncio ? (Haven't found any)

> What's a good strategy to migrate a larger python codebase towards asyncio ? (Haven't found any)

use gevent instead

Re: Waiting in Asyncio

#28

Post by ghostwriter »

hynek wrote: post #17
dilandau wrote: post #2

All these subtleties and gotchas signify, to me, a shit implementation and a fragmented interface. The exact opposite of the Zen of python.

A more charitable view is that they signify an API that evolved from callback-based APIs like Twisted over many years and made nicer APIs like curio or trio only possible (whose features/insights slowly feed back into asyncio but backward compat keeps the warts around).

What's the added value of curio and trio APIs, compared to gevent's Queues, Pools, and Groups?

https://sdiehl.github.io/gevent-tutorial/#data-structures

Re: Waiting in Asyncio

#29

Post by uranusjr »

j88439h84 wrote: post #9
hedora wrote: post #4

Earlier quoted context omitted.

Can you give a practical example of a library or python subsystem that lives up to the “Zen of python” you are referring to?

There are lots. `requests` is the most famous.

requests is “clean enough” if you only use the highest level functionalities like making stateless requests and read out the response. I wouldn’t say it fits the description once you go past that.

Zen of Python is an illusion. Complex things are complex, there’s simply no way to have an interface that’s always clean, useful, and maintainable.

Re: Waiting in Asyncio

#30

Post by j88439h84 »

uranusjr wrote: post #29
j88439h84 wrote: post #9

Earlier quoted context omitted.

There are lots. `requests` is the most famous.

requests is “clean enough” if you only use the highest level functionalities like making stateless requests and read out the response. I wouldn’t say it fits the description once you go past that. Zen of Python is an illusion. Complex things are complex, there’s simply no way to have an interface that’s always clean, useful, and maintainable.

These are about priorities, not about achieving perfection. It is easy to distinguish different pieces of software on these dimensions. For example:

"Beautiful is better than ugly." Requests is more beautiful than urllib2.

"Simple is better than complex." Trio is simpler than asyncio.

"Readability counts." Python is more readable than C++.

"There should be one-- and preferably only one --obvious way to do it." ''.join(strings), not sum(strings, '').

Note what isn't in the Zen of Python: performance, close-to-the-metal-ness, correctness, portability.

Python often fails to live up to its values, but that doesn't make them meaningless.

Post reply on HN