Earlier quoted context omitted.
> The pattern for fire-and-forget is impossible Good, that's an antipattern in the coroutines concurrency model.
Then someone should really update the official python docs that explain the fire-and-forget pattern ( https://docs.python.org/3/library/asyncio-task.html#asyncio.... )! I had a FastAPI server, and calling a particular endpoint is supposed to kick off some work in the background. The background work does very little CPU work, but does often need to await more work for several minutes, so it's a good fit for asyncio. H…
Regarding the fire-and-forget pattern I can think of at least two issues, let me illustrate them with the following example:
import asyncio
from typing import Any, Coroutine
_background_tasks = set[asyncio.Task[None]]()
def _fire_and_forget(coro: Coroutine[Any, Any, None]) -> None:
task = asyncio.create_task(coro)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
async def _run_a() -> None: # to illustrate issue A
raise RuntimeError()
async def _run_b() -> None: # to illustrate issue B
await asyncio.sleep(1)
print('done')
async def main() -> None:
# issue A: Exceptions can't be caught
try:
_fire_and_forget(_run_a())
except RuntimeError as exc:
print(f'Exception caught: {exc}')
# issue B: Task won't complete
_fire_and_forget(_run_b())
if __name__ == '__main__':
asyncio.run(main())
For issue A: Any raised error in the background task can't be caught and will crash the running main thread (the process)
For issue B: Background tasks won't be completed if the main thread comes to a halt
With the decision for the fire-and-forget pattern you'll make a deliberate choice to leave any control of the runtime up to blind chance. So from an engineering POV that pattern isn't a solution-pattern to some real problem, it's rather a problem-pattern that demands a reworked solution.
> How do you want it to be structured
Take a look at the caveats for FastAPI/Starlette Background Tasks: https://fastapi.tiangolo.com/tutorial/background-tasks/#cave...
Losing control of a background task (and therefor the runtime) might be fine for some demo project, but I think you'll want to notice raised errors in any serious production system, especially for any work that takes several minutes to complete.