I've been using https://www.pgflow.dev for workflows which is built on pgmq and am really impressed so far. Most of the logic is in the database so I'm considering building an Elixir adapter DSL.
Build durable workflows with Postgres
31–40 of 58 posts
Re: Build durable workflows with Postgres
#32I've been looking at migrating to Temporal, but this looks interesting. For context, we have a simple (read: home-built) "durable" worker setup that uses BullMQ for scheduling/queueing, but all of the actual jobs are Postgres-based. Due to the cron-nature of the many disparate jobs (bespoke AI-native workflows), we have workers that scale up/down basically on the hour, every hour. Temporal is the obvious solution, bu…
I'd love to learn more about what you're building--just reach out at peter.kraft@dbos.dev. One option is that you have DBOS workflows that schedule and submit jobs to an external worker app. Another option is that your workers use DBOS queues ( https://docs.dbos.dev/python/tutorials/queue-tutorial ). I'd have to better understand your use case to figure out what would be the best fit.
Do you think an app’s (e.g. FastAPI) backend should be the DBOS Client, submitting workflows to the DBOS instance? And then we can have multiple DBOS instances with each picking up jobs from a queue?
Re: Build durable workflows with Postgres
#33Earlier quoted context omitted.
I'd love to learn more about what you're building--just reach out at peter.kraft@dbos.dev. One option is that you have DBOS workflows that schedule and submit jobs to an external worker app. Another option is that your workers use DBOS queues ( https://docs.dbos.dev/python/tutorials/queue-tutorial ). I'd have to better understand your use case to figure out what would be the best fit.
I’m also interested in what you think can become best practices where we can have (auto-scaling) worker instances that can pick up DBOS workflows and execute them. Do you think an app’s (e.g. FastAPI) backend should be the DBOS Client, submitting workflows to the DBOS instance? And then we can have multiple DBOS instances with each picking up jobs from a queue?
Queue docs: https://docs.dbos.dev/python/tutorials/queue-tutorial Client docs: https://docs.dbos.dev/python/reference/client
Re: Build durable workflows with Postgres
#34Recently moved some of the background jobs from graphile worker to DBOS. Really recommend for the simplicity. Took me half an hour. I evaluated temporal, trigger, cloudflare workflows (highly not recommended), etc and this was the easiest to implement incrementally. Didn't need to change our infrastructure at all. Just plugged the worker where I had graphile worker. The hosted service UX and frontend can use a lot of…
Re: Build durable workflows with Postgres
#35Re: Build durable workflows with Postgres
#36How does this compare with inngest or restate? We currently use inngest right now and it works great but the typescript API is a bit clunky
Here's a blog post explaining the DBOS architecture in more detail: https://www.dbos.dev/blog/what-is-lightweight-durable-execut...
Here's a comparison with Temporal, which is architecturally similar to Restate and Inngest: https://www.dbos.dev/blog/durable-execution-coding-compariso...
Re: Build durable workflows with Postgres
#37Earlier quoted context omitted.
Both do durable workflows with similar guarantees. The big difference is that DBOS is an open-source library you can add to your existing code and run anywhere, whereas Durable Functions is a cloud offering for orchestrating serverless functions on Azure.
As far as I know, Azure Durable Functions doesn't have a server-side proprietary component and it's actually fully open source framework/clients as well. So it's actually not a cloud offering per-se. You can see the full implementations at: * https://github.com/Azure/durabletask * https://github.com/microsoft/durabletask-go
Re: Build durable workflows with Postgres
#38This is how you write a technical article. Thanks to the author for the nice read :)
Re: Build durable workflows with Postgres
#39Recently moved some of the background jobs from graphile worker to DBOS. Really recommend for the simplicity. Took me half an hour. I evaluated temporal, trigger, cloudflare workflows (highly not recommended), etc and this was the easiest to implement incrementally. Didn't need to change our infrastructure at all. Just plugged the worker where I had graphile worker. The hosted service UX and frontend can use a lot of…
Re: Build durable workflows with Postgres
#40Recently moved some of the background jobs from graphile worker to DBOS. Really recommend for the simplicity. Took me half an hour. I evaluated temporal, trigger, cloudflare workflows (highly not recommended), etc and this was the easiest to implement incrementally. Didn't need to change our infrastructure at all. Just plugged the worker where I had graphile worker. The hosted service UX and frontend can use a lot of…
Agree on the UI - I wish it was improved