My first endeavor with asyncio felt worse than a beating with a wet rubber hose. But it was a character building experience to really get up close and personal with the asynchronous model and I can definitely see its advantages over imperative, as well as it's always good to have another tool in the box.
The magic of asyncio explained
81–90 of 95 posts
Re: The magic of asyncio explained
#82Wasn't there a language where every call was async? Instead of async ... A/returing Future[A] it did/would return A from method calls. If it didn't exist, one can imagine one. A.x = 3 would be wrapped in A.map(_.x = 3) etc. So you write code that would be executed when you finally await a value. No more red/blue world. Would probably need coroutines instead of threads for executing.
Just to post it as an answer instead of a question. That's Haskell's IO.
It is just one of the lots of concurrency behaviors available in libraries. Also, parallelism is "free" on pure code.
Re: The magic of asyncio explained
#83Earlier quoted context omitted.
It's hilarious when a single comment on HN opens asyncio more than the tutorial being discussed.
Well, honestly, I think most HN readers go to the comments before the actual article. I know I do. The whole value of this website is that we got 1000 of experts in their fields, ready to give you their insight.
HN isn’t what it was a few years ago, but it’s still a hell of a lot better than Hackernoon.
Re: The magic of asyncio explained
#84Python is my language of first choice, but I must say that I am not that thrilled how this multithreading ended up. There are many tutorials about the topic promising to explain how it works, usually in the form of "simple introduction". But when one tries to implement something production-ready, with correct error handling etc., things starts to complicate pretty quickly; at least that was my experience. I don't wan…
That's because most of those tutorials have not been written by somebody actually putting something in production. I've been using asyncio for a while now, and you can't get away with a short introduction since: - it's very low level - it's full of design flaws and already has accumulated technical debt - it requires very specific best practices to be usable I'm not going to write a tutorial here, it would take me a…
> - don't pass the loop around. Use asyncio.get_event_loop(). This way your code will be independent of the loop creation process.
Eh. I've been passing the loop around as an optional kw argument in most of my code...
The idea was for the code not to depend on a global somewhere (I hate globals) and to "be sure" the loop used among all the code was the same, unless explicitly passed. Of course I never used that "feature". I thought I read this somewhere when I was looking up at Twisted and they were saying to pass it explicitly, but I'm not so sure now...
Re: The magic of asyncio explained
#85Python is my language of first choice, but I must say that I am not that thrilled how this multithreading ended up. There are many tutorials about the topic promising to explain how it works, usually in the form of "simple introduction". But when one tries to implement something production-ready, with correct error handling etc., things starts to complicate pretty quickly; at least that was my experience. I don't wan…
That's because most of those tutorials have not been written by somebody actually putting something in production. I've been using asyncio for a while now, and you can't get away with a short introduction since: - it's very low level - it's full of design flaws and already has accumulated technical debt - it requires very specific best practices to be usable I'm not going to write a tutorial here, it would take me a…
Good information, but it all depends on use case, for example I use a lot of await and "async with" in coroutines.
Then start tasks as:
tasks = [coroutine(i) for i in parameters]
for task in asyncio.as_completed(tasks)
There are many ways of using it.
Re: The magic of asyncio explained
#86Earlier quoted context omitted.
That's because most of those tutorials have not been written by somebody actually putting something in production. I've been using asyncio for a while now, and you can't get away with a short introduction since: - it's very low level - it's full of design flaws and already has accumulated technical debt - it requires very specific best practices to be usable I'm not going to write a tutorial here, it would take me a…
> - don't pass the loop around. Use asyncio.get_event_loop(). This way your code will be independent of the loop creation process. Eh. I've been passing the loop around as an optional kw argument in most of my code... The idea was for the code not to depend on a global somewhere (I hate globals) and to "be sure" the loop used among all the code was the same, unless explicitly passed. Of course I never used that "feat…
You supposed to have only a single event loop per thread, the standard event loop policy ensures that the value is thread local (you can change that by modifying the policy), unless you're doing something unusual with multiple loops in the same thread you will never need to pass the value.
Also if you are passing the loop and are doing multi threading, you need to be careful, because if you pass it to another thread you might see weird issues.
I initially also started explicitly pass loop around but once decided to combine asyncio with threads I realized that it is better to trust get_event_loop() to do the job correctly. The only exception is when I need to schedule coroutine in one thread for another thread. In that case I need loop from a different thread so I can invoke call_soon_threadsafe().
Re: The magic of asyncio explained
#87Earlier quoted context omitted.
That's because most of those tutorials have not been written by somebody actually putting something in production. I've been using asyncio for a while now, and you can't get away with a short introduction since: - it's very low level - it's full of design flaws and already has accumulated technical debt - it requires very specific best practices to be usable I'm not going to write a tutorial here, it would take me a…
Wow, really nice list, I wish I knew it before I started to work with asyncio. > stay sane, you should never, ever, have an dangling awaitable anywhere. Always get a reference on all your awaitables. Decide where in the code you think their life should end. This is the most difficult part for me, it's not trivial to know if a function you're calling is async or not without looking at the function source, specially wh…
Anything that's defined as "async def" and that you call with await and friends should be async.
Yes, it's possible to write coroutine and use "async def" without any await inside, but in those cases the library authors should just made it a normal function.
I would say that this is a bug in the library.
Re: The magic of asyncio explained
#88Yet another asyncio tutorial that shows you to run a few sleep tasks concurrently. Can we finally get one that shows how to do real stuff such like socket programming, wrapping non-async-compatible libraries and separating cpu-intensive blocking tasks to awaitable threads?
> such like socket programming That's one of my biggest pet peeves (and if you see my other comments, you'll notice I have quite a few). To do socket programming in asyncio, you can either use: - protocols, with a nice reusable API and an interface that clearly tells you where to do what. But you can't use "await". You are back to creating futures and attaching callback like 10 years ago. - streams, where you can use…
> If I wanted extreme perfs I'd use C. If I wanted extreme pureness I'd use Haskell.
Haskell concurrent socketry is decent:
https://kyle.marek-spartz.org/posts/2014-08-26-concurrent-im...
Re: The magic of asyncio explained
#89Wasn't there a language where every call was async? Instead of async ... A/returing Future[A] it did/would return A from method calls. If it didn't exist, one can imagine one. A.x = 3 would be wrapped in A.map(_.x = 3) etc. So you write code that would be executed when you finally await a value. No more red/blue world. Would probably need coroutines instead of threads for executing.
Just to post it as an answer instead of a question. That's Haskell's IO. It is just one of the lots of concurrency behaviors available in libraries. Also, parallelism is "free" on pure code.
From my understanding, this is not Haskells IO - though my time with Haskell is limited.
1. Haskell uses special notation 'do' to handle access to IO wrapped values, e.g. (contrieved example, one would not use do for such simple cases)
y = do x y = x + 1
doSomething :: Int -> IO Int
def doSomething(i:Int):Int
type Result a = ReaderT Env ( ErrorT String ( StateT Integer Identity))
4. This the same as my Scala code, where I have cats FutureT monad transformers with Scalactic errors OrT Every stacks showing up all over my APIs as a type alias of 'WithErrors'.
5. 'Also, parallelism is "free" on pure code.' Not sure what's that got to do with it, but yes if you have no concurrency problems (concurrent writes to shared data) you don't need to think about concurrency and parallelism is free.
But if my understanding is wrong, I'm happy to learn something about concurrency in Haskell without it showing in code and type signatures.
Re: The magic of asyncio explained
#90Earlier quoted context omitted.
They're sort of similar, and you can probably get the same work done in either system, but I think real threading (green or otherwise), may leave you with less cognitive load. Spawning a thread may be complex, and thinking about how the threads are scheduled is often complex, but what each thread does can be very simple -- and you don't have to think about 'long running things need to be futured/awaited', you just do…
Python has proper threads, and they're anything but simple.
Python threads aren't simple, because of the shared everything model python uses, so any variable access requires the GIL. Shared nothing threads are much simpler to work with.