Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building durable workflows on Postgres

dbos.dev

Post reply on HN
Print view View original

61–70 of 159 posts

Re: Building durable workflows on Postgres

#61

Post by llimllib »

vrm wrote: post #51
llimllib wrote: post #26

Armin Ronacher's `absurd` is an implementation of durable workflows for postgres: https://lucumr.pocoo.org/2025/11/3/absurd-workflows/ https://github.com/earendil-works/absurd https://earendil-works.github.io/absurd/ I've not used it, but it's worth comparing to other options

If you don't need a ton of throughput I think `absurd` (and our Rust derivative `durable`) are very nice options that keep the client side extremely simple. It's also lightweight enough that a coding agent can keep the entire thing in its head easily and just run queries to look up state as needed.

cross-checking your profile suggests that https://github.com/tensorzero/durable is the repo you're referring to

You might consider another name for it, that one is wholly ungoogle-able! Looks neat though

Re: Building durable workflows on Postgres

#62

Post by sroussey »

buremba wrote: post #27

All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, y…

Use different “databases” besides public at the very start. No joins between them. You will be in a good position to just split the postgres instance by those at a later date. They will have different usage patterns than the merged version you have now, and will be easier to optimize and will buy you some time. And time is all you need.

Re: Building durable workflows on Postgres

#63

Post by dieselgate »

tibbon wrote: post #52
buremba wrote: post #27

All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, y…

But when you hit that wall, it is hard to stop and convince people to use different patterns and systems. I've seen so many tables go from "it will only be a few thousand rows" to suddenly several TB and then people are looking confused when performance and db admin tasks get really difficult. I'm working at a scale where almost every day I have to ask people "are you use you need to treat that as relational data? It…

> are you use you need to treat that as relational data?

Is this intended to be "you sure you need..."?

Re: Building durable workflows on Postgres

#64

Post by cultofmetatron »

buremba wrote: post #27

All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, y…

conversely, startups that start scaling for tbs of data never make it to needing tbs of data. They burn too much energy scaling when they don't yet have a product people want yet.

Re: Building durable workflows on Postgres

#65

Post by agumonkey »

tibbar wrote: post #56
stuartaxelowen wrote: post #40

My dream is, instead of separating data storage, state machines, valid state constraints, and the logic that transitions between valid states, we can actually unify these into some kernel of app state. Honestly, Postgres already has a lot of these capabilities, but I don’t see an obvious story on the app or product level, providing provably correct sets of states that apps can transition between, and which they can a…

This has been tried, but thousand-line stored procedures are truly a nightmare.

was it due to the language expressiveness forcing too much verbosity ? (honest question)

Re: Building durable workflows on Postgres

#66

Post by ceres »

buremba wrote: post #27

All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, y…

Just an fyi, when I try to sign in with google for your app I get the message: "The app is requesting access to sensitive info in your Google Account. Until the developer (*reka*kc*@gmail.com) verifies this app with Google, you shouldn't use it."

Re: Building durable workflows on Postgres

#68

Post by vrm »

llimllib wrote: post #61
vrm wrote: post #51

Earlier quoted context omitted.

If you don't need a ton of throughput I think `absurd` (and our Rust derivative `durable`) are very nice options that keep the client side extremely simple. It's also lightweight enough that a coding agent can keep the entire thing in its head easily and just run queries to look up state as needed.

cross-checking your profile suggests that https://github.com/tensorzero/durable is the repo you're referring to You might consider another name for it, that one is wholly ungoogle-able! Looks neat though

TBH it's intended only for internal use (we don't even publish it as a crate at this point) so I don't particularly mind it being low-key. But I appreciate it!

Re: Building durable workflows on Postgres

#69

Post by turtlebits »

lll-o-lll wrote: post #57
temporal_thr123 wrote: post #16

Earlier quoted context omitted.

I run a large on-prem temporal setup - throwaway acct as they will likely out me. Temporal is, in my opinion having run it in prod for over a year - poorly designed, slow and ridicliously heavy infra wise. If you're doing anything non-trivial (say, 200+ events/workflow) and you need to run only a couple hundred of them concurrently all day, you're going to spend millions on infra, and it's still going to absolutely s…

> If you're doing anything non-trivial (say, 200+ events/workflow) and you need to run only a couple hundred of them concurrently all day, you're going to spend millions on infra, and it's still going to absolutely suck. Where are the “millions” on infra going? It’s a handful of services and a Postgres? > Their sales team is also absolutely appalling and desperate. You said “on-prem”. It’s open source; why are you de…

The same with any "open-source" enterprise ($$$) software. It sucks to run yourself. Docs on running/errors are non-existent. Their helm charts are broken. Instead of degraded performance, it just fails.

Re: Building durable workflows on Postgres

#70

Post by jhot »

throwaw12 wrote: post #4

Curious to know experience of people using DBOS and Temporal. I have used Temporal in the past, works really good, my only problem with it was some limits on request payload or event sizes, created some inconveniences to us when building solutions. It also enforces good engineering practices, but sometimes you don't want to write special logic if your CSV file is larger than 2Mb, upload it to S3, pass link, then down…

Haven't used DBOS but use Temporal at current job and used it at previous job as well so I have about 1.5 years under me now. I also run it at home to handle some home automation tasks that aren't super time sensitive (the latency of workflows isn't super bad, but I wouldn't use one for something that is triggered by a motion event in my house unless we're talking about a timeout to turn something off after inactivity).

I really like running a thin rest API in front of it inside your vpc or k8s cluster or whatever to help with event driven triggers so that they don't have to worry about Temporal auth and checking workflow status if there is any decision making around that. This helps keep your event as logic-free as possible.

Let me give a vague example: you have some sort of db trigger, and this trigger either acts directly or puts the event on a queue, your handler calls the thin rest api with the necessary event details, rest API can make the decision if this starts a workflow, signals an existing one, or ignores it (the pattern for this can vary based on the situation, but SignalWithStart is common for me or just dropping if the event is not worthy of starting a workflow and no workflow for that exists).

Then the parent/child workflow ability is very valuable when you need to orchestrate different self-contained behaviors for a single object's lifecycle, with cancellability when an external factor changes the trajectory of an object.

Long, vague story short, I find it very powerful and easy to work with and has really helped move lifecycle logic out of APIs where things can easily become riddled with debt and precarious to manage. I agree with you that it helps follow more best-practices instead of just throwing logic some place that seems easy but becomes a hidden trap later.

Post reply on HN