Viewing profile — hmaxdml

I thought that fell under BYOD (device) :)

What's reasonable? DBOS has users running queues at millions of tasks per hour. For fair queuing you can have partitioned queues where only active partitions consume resources

Not saying this in a mean way: It seems to me this can be boiled down to "using things without understanding how they work doesn't scale". Yes, vanilla listen/notify doesn't scale.…

The database is exactly the hardcore piece of engineering that's been designed to scale and be fault tolerant for decades

Because you likely already have a database and likely don't need to bring on an entire new distributed system to orchestrate your workflows.

A PG-backed queue is in code right after being in PG, and the beauty of a neat durable queue framework is in exposing it conveniently and efficiently.

Postgres does scale pretty well: https://www.dbos.dev/blog/benchmarking-workflow-execution-sc... Tens of thousands of workflows per second

That's why their entire business model -- like Astronomer's -- is geared toward cloud hosting. The architecture is so complex it takes a full time team to run it.

Have you looked into DBOS? Same thesis: durable and reliable workflows are hard to manage -- it just doesn't have to be as hard as Temporal makes it be :)

DBOS python supports SQLite. Go is supporting it next release

I've talked to dozens of engineers who built their home grown "durable" stack. Most of them eventually moved on to buying vs building, when their system actually scaled. It's just …

Yeah, we've observed that too: people start implementing their own retry logic, idempotency, etc. But then they grow a hard to maintain, complex stack that's not their core busines…

Listen/notify is poised to become much better in PG 18 and 19

As you said, the example is simple and it might not be obvious to people without prod experience what the problems can be. Postgres can give you all the primitives you need to solv…

We've found that durable workflows is a much needed primitive for agents control flow. They give a structure for deterministic replays, observability, and, of course, fault toleran…

Automatic crash detection for your process is built-in our Conductor offering. The library has a default recovery mode when used standalone. What do you find strange with workflow …

I notice you didn't provide any specific comparison alongside that comment, which makes me feel frustrated because I think the Temporal workflow SDK is very different. Architectura…

Thanks for the comment (author here). I wanted this post to focus on the Golang specific implementation, not dwell on the durable execution ecosystem at large. With respect to cont…

Conductor is about enterprise features like automatic workflow recovery, alerting, or RBAC. The GUI is a nice to have -- but all your workflow data are in Postgres. You can access …

Durable execution has already been mentioned as the existing solution for this problem, but I would like to call out a specific pattern that DE makes obsolete: the outbox pattern. …

These are all important concerns, but I'd go for an off the shelf library that does it for me (disclaimer I work at https://github.com/dbos-inc )

The hype is because DE is such an dev exp improvement over building your own queue. Good DE frameworks come with workflows, pub/sub, notifications, distributed queues with tons of …