pg_durable: Microsoft open sources in-database durable execution
1–10 of 119 posts
Re: pg_durable: Microsoft open sources in-database durable execution
#22026 is the year of the Postgres queue! (DBOS[0], pgQue[1]) It's awesome that the community is contributing this and giving us the option to use it.
As an ex-app engineer though, I kind of prefer my queue logic to be in code, in Git, but maybe with the right tooling, you can change my mind. :)
Re: pg_durable: Microsoft open sources in-database durable execution
#3This feels like the wrong solution to an age old problem solved by the DAG schedulers like Apache Airflow for a while now.
Why would I want to store my control flow in the database and not in code? It feels strange.
Not trying to dismiss the project, I'm just not getting it yet I think.
Re: pg_durable: Microsoft open sources in-database durable execution
#4> When not to use it
> …
> The workflow mostly lives outside Postgres and spans many heterogeneous systems.
How is this project at all comparable to something like Temporal? Am I misunderstanding the limitation implied by this particular recommendation?
Re: pg_durable: Microsoft open sources in-database durable execution
#5Looks pretty good but I wonder why they didn’t build it on pgmq? If you’re on elixir I maintain a DAG package around this (based on and compatible with pgflow.dev which is TS/Deno).
Re: pg_durable: Microsoft open sources in-database durable execution
#6> When not to use it > … > The workflow mostly lives outside Postgres and spans many heterogeneous systems. How is this project at all comparable to something like Temporal? Am I misunderstanding the limitation implied by this particular recommendation?
I aggree - I'm not understanding the value of the project either if you look at the example here https://github.com/microsoft/pg_durable/blob/main/examples/i...
It's an interesting technical achievement I guess, but it's very bizarre to try and read this
SELECT df.start(
@> (
($$SELECT ... FROM demo.invoices WHERE status = 'pending'$$ |=> 'inv')
~> df.if_rows('inv',
$$UPDATE ... SET status = 'processing'$$
~> (df.http(...) |=> 'resp')
~> df.if($$SELECT $r.ok$$,
-- classify, branch, wait for signal ...
),
df.sleep(5)
)
),
'invoice-approval-pipeline'
);Re: pg_durable: Microsoft open sources in-database durable execution
#7If understanding correctly, Absurd (by the Pi LLM harness devs) minimizes the pure db approach as much as possible. I only just started getting into the topic myself, though.
Re: pg_durable: Microsoft open sources in-database durable execution
#8Can anyone explain why I would want to use this over an orchestration tool that lives outside the DB? Read through the Readme and some of the examples, I still don't get it.
Re: pg_durable: Microsoft open sources in-database durable execution
#9Looks pretty good but I wonder why they didn’t build it on pgmq? If you’re on elixir I maintain a DAG package around this (based on and compatible with pgflow.dev which is TS/Deno). https://github.com/agoodway/pgflow
(pg_durable committer here)
The provider is an extensibility point. We just shipped the simplest version of it. Happy to take contribs if someone sends a pgmq based provider!
Re: pg_durable: Microsoft open sources in-database durable execution
#10Can anyone explain why I would want to use this over an orchestration tool that lives outside the DB? Read through the Readme and some of the examples, I still don't get it.
It’s sometimes convenient if database is the only ”stateful” component in architecture.
Also if all the "state" is in one database, then you have better chance of getting consistent backups.