Temporal Python – A durable, distributed asyncio event loop (2023)
1–10 of 56 posts
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#2Credit: @kodablah and @chippiewill, thanks for turning me into this!
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#3As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker?
> See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal.
That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why every third party async Python library/framework feels the need to take over the default event loop instead of just building on top of it.
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#4As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker? > See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal. That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why ev…
I think it means that your code could be resumed on a different machine.
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#5As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker? > See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal. That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why ev…
Temporal is completely above and beyond Asyncio. It's a full scheduling of work and queues that's cross-machine, cross-language, and very transparent.
A workflow is the code that handles only deterministic actions and calls activities.
Activities are functions that do anything you want, typically affecting other systems with network or file calls.
A worker is the running process connected to Temporal with registered workflows & activities for it to pass work to.
I'm doing a lot of work with alert handling and provisioning systems using Temporal. Temporal in two minutes is a great video explanation: https://www.youtube.com/watch?v=f-18XztyN6c
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#6As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker? > See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal. That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why ev…
The thing with event loops in python is that they are not a single, all-governing scheduler (as e.g. in the BEAM).
ev loops instead are a mid-layer concept that sits below other infrastructure such as threads and processes. And (perhaps somewhat frustratingly) it is not too uncommon to have multiple ev loops in parallel. See for example the proxy.py project, which offers to run one async loop per process for a speedup.
As a result, there are some incentives to swap out the loop itself, e.g. for faster implementations like uvloop, because they are somewhat pluggable anyways.
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#7Relying on external APIs or databases within activities might lead to variability in workflow execution.
Also, on handling HTTP errors in activities by raising an "ApplicationError" based on the status code, might simplifies error handling but might need to see how it accounts for more complex scenarios where errors are transient or where a retry could be successful even for some client errors like rate limiting or temporary unavailability etc.
As the asyncio library itself does have a steep learning curve, integration of asyncio with workflow systems like Temporal that also uses Pythons native asynchronous features, developers should be careful about indirect or subtle bugs, especially in error handling and task management.
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#8As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker? > See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal. That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why ev…
The thing with event loops in python is that they are not a single, all-governing scheduler (as e.g. in the BEAM). ev loops instead are a mid-layer concept that sits below other infrastructure such as threads and processes. And (perhaps somewhat frustratingly) it is not too uncommon to have multiple ev loops in parallel. See for example the proxy.py project, which offers to run one async loop per process for a speedu…
> all-governing scheduler (as e.g. in the BEAM).
Does this also mean they have preemptive multitasking like in BEAM?
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#9As someone familiar with asyncio, I don't understand what this is or what it's for. What's an activity, workflow, or worker? > See the asyncio.sleep in there? That’s no normal local-process sleep; that’s a durable timer backed by Temporal. That's the normal asyncio.sleep. What does backed by Temporal mean? Reading further, it appears that Temporal is replacing the default asyncio event loop. I don't understand why ev…
I just had to write a Python program that handled multiple async events - from a serial line, and with a tkinter GUI. The only way to make it 'truly' async was to handle the runloops myself, and add a separate queue, onto which I push coroutines, for processing. UI events and Serial I/O events (involving passing of messages to update states on both sides) all have to be pushed through the same mechanism in order to gain the functionality I need.
Sure, I 'could just use asyncio', or somehow work out how to crowbar serial i/o into tkinters' runloop. But in the end, writing my own just made more sense, and more importantly: it works great. I can have serial i/o and UI acting independently, but coordinating through a single queue .. this works so well.
>I don't understand why every third party async Python library/framework feels the need to take over the default event loop instead of just building on top of it.
Because you don't always have what you need to get the crowbar in place, nor big enough leverage to make space for what you have to do, asynchronously, in the app.
Re: Temporal Python – A durable, distributed asyncio event loop (2023)
#10Isn't this just threads but with more surprise gotos?