Live data from Hacker News

Show HN: DBOS Java – Postgres-Backed Durable Workflows

github.com

41–50 of 59 posts

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#41
post #4

We are increasingly using Temporal with Temporal Cloud and soon Nexus to manage numerous workflows. I'm curious what type of observability is avialable for DBOS and how much of that you get for "free". The reason we ended up in Temporal was not that previous job-systems were unreliable, it was simply that nobody wanted to go dig through a database to find out what happened with their job, and nobody has time/energy t…

There's an observability and workflow management UI: https://docs.dbos.dev/java/tutorials/workflow-management You can view your workflows and queues, search/filter them by any number of criteria, visualize graphs of workflow steps, cancel workflows, resume workflows, restart workflows from a specific step--everything you'd want. Currently, this is available as a managed offering (Conductor - https://docs.dbos.dev/pro…

I was super interested in DBOS but I had to back out when I figured that the observability isn't self hostable yet :(, so I'm chuffed to hear its coming!

Whats the best way to hear about it when it does? Maybe newsletter I can register to or something.

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#42
Hello Peter. Thank you for your work. I really like this approach. I too have been following Temporal and I like it, but I don't think it's a good match for simpler systems.

I've been reading the DBOS Java documentation and have some questions, if you don't mind:

- Versioning; from the looks of it, it's either automatically derived from the source code (presumably bytecode?), or explicitly set for the entire application? This would be too coarse. I don't see auto-configuration working, as a small bug fix would invalidate the version. (Could be less of a problem if you're looking only at method signatures... perhaps add to the documentation?) Similarly, for the explicit setting, a change to the version number would invalidate all existing workflows, which would be cumbersome.

Have you considered relying on serialVersionUID? Or at least allowing explicit configuration on a per workflow basis? Or a fallback method to be invoked if the class signatures change in a backward incompatible way?

Overall, it looks like DBOS is fairly easy to pick up, but having a good story for workflow evolution is going to be essential for adoption. For example, if I have long-running workflows... do I have to keep the old code running until all old instances complete? Is that the idea?

- Transactional use; would it be possible to create a new workflow instance transactionally? If I am using the same database for DBOS and my application, I'd expect my app to do some work and create some jobs for later, reusing my transaction. Similarly, maybe when the jobs are running, I'd perhaps want to use the same transaction? As in, the work is done and then DBOS commits?

I know using the same transaction for both purposes could be tricky. I have, in fact, in the past, used two for job handling. Some guidance in the documentation would be very useful to have.

Thank you.

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#43

Looks great, shame that due to annotation-based API it's gonna be a pain to use in Clojure.

One thing we're looking at right now is what it would take to support Clojure or Kotlin.

Fully declarative approach is the way to go for both IMHO. I'm more familiar with Kotlin than Kotlin. But its coroutines framework and structured concurrency are well aligned with something like this. Essentially you are doing is kind of a stateful form of structured concurrency where state is preserved in a DB and resilient against sub task failures, nodes dying, etc.

This reminds me of a product called restate. I talked to some people in that company a while ago. Their solution is built in Rust I think but they have clients for all sorts of platforms. Including Kotlin. Cool company and distributed workflow engines and agentic / long running workflows feel like a good match.

There are lots of other solutions in this space. I believe an ex Red Hat person is working on rebooting a workflow engine called Kogito based on something that orginally lived under their umbrella.

There's a long history of very enterprisy business process management stuff here. Lots of potential for overengineered solutions.

I once got sucked into a Spring Batch centric project and it was hopelessly overengineered for the requirements. Gave me a proper headache. Nothing was simple. Everything was littered in magic annotations causing all sorts of weird side effects. That's why I prefer declarative approaches with simple functions. Which is what the Kotlin syntax enables relative to Java. You can do the same technically in java but it quickly becomes an unreadable mess of function chaining.

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#45

The Java version looks pretty danged cool paging through the some of the code. I was trying to think of a use case for this and I was reminded of a Sun Microsystems demo (yes I'm that old) I saw. The paused a JVM and "slept" it, then kicked up on another machine almost instantly, all over the network. Was a pretty cool party trick, but then they did it when an HTTP request came in (Serverless was invented a LONG time…

There is a modern equivalent based on Linux criu.

https://github.com/CRaC/

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#47
post #42

Hello Peter. Thank you for your work. I really like this approach. I too have been following Temporal and I like it, but I don't think it's a good match for simpler systems. I've been reading the DBOS Java documentation and have some questions, if you don't mind: - Versioning; from the looks of it, it's either automatically derived from the source code (presumably bytecode?), or explicitly set for the entire applicat…

Thanks for the great questions!

1. Yes, currently versioning is either automatically derived from a bytecode hash or manually set. The intended upgrade mechanism is blue-green deployments where old workflows drain on old code versions while new workflows start on new code versions (you can also manually fork workflows to new code versions if they're compatible). Docs: https://docs.dbos.dev/java/tutorials/workflow-tutorial#workf...

That said, we're working on improvements here--we want it to be easier to upgrade your workflows without blue-green deployments to simplify operating really long-running (weeks-months) workflows.

2. Could I ask what the intended use case is? DBOS workflow creation is idempotent and workflows always recover from the last completed step, so a workflow that goes "database transaction" -> "enqueue child workflow" offers the same atomicity guarantees as a workflow that does "enqueue child workflow as part of database transaction", but the former is (as you say) much simpler to implement.

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#48
post #44

I remember reading that restate.dev is a 'push' based workflow and therefore works well with serverless workflows: https://news.ycombinator.com/item?id=40660568 what is your input on these two topics? aka pull vs push and working well with serverless workflows

The Restate model depends on a long-running external orchestrator to do the "pushing". However, that comes with downsides--you have to operate that orchestrator and its data store in production (and it's a single point of failure) and you have to rearchitect your application around it.

DBOS implements a simpler library-based architecture where each of your processes independently "pulls" from a database queue. To make this work in a serverless setting, we recommend using a cron to periodically launch serverless workers that run for as long as there are workflows in the queue. If a worker times out, the next will automatically resume its workflows from their last completed step. This Github discussion has more details: https://github.com/dbos-inc/dbos-transact-ts/issues/1115

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#49
post #42

Hello Peter. Thank you for your work. I really like this approach. I too have been following Temporal and I like it, but I don't think it's a good match for simpler systems. I've been reading the DBOS Java documentation and have some questions, if you don't mind: - Versioning; from the looks of it, it's either automatically derived from the source code (presumably bytecode?), or explicitly set for the entire applicat…

Thanks for the great questions! 1. Yes, currently versioning is either automatically derived from a bytecode hash or manually set. The intended upgrade mechanism is blue-green deployments where old workflows drain on old code versions while new workflows start on new code versions (you can also manually fork workflows to new code versions if they're compatible). Docs: https://docs.dbos.dev/java/tutorials/workflow-tut…

> versioning

Here's an example of a common long-running workflow: SaaS trials. Upon trial start, create a workflow to send the customer onboarding messages, possibly inspecting the account state to influence what is sent, and finally close the account that hasn't converted. This will usually take 14-30 days, but could go for months if manually extended (as many organisations are very slow to move).

I think an "escape hatch" would be useful here. On version mismatch, invoke a special function that is given access to the workflow state and let it update the state to align it with the most recent code version.

> transactional enqueuing

A workflow that goes "database transaction" -> "enqueue child workflow" is not safe if the connection with DBOS is lost before the second step completes safely. This would make DBOS unreliable. Doing it the other way round can work provided each workflow checks that the "object" to which it's connected exists. That would deal with the situation where a workflow is created, but the transaction rolls back.

If both the app and DBOS work off the same database connection, you can offer exactly-once guarantees. Otherwise, the guarantees will depend on who commits first.

Personally, I would prefer all work to be done from the same connection so that I can have the benefit of transactions. To me, that's the main draw of DBOS :)

Re: Show HN: DBOS Java – Postgres-Backed Durable Workflows

#50

I like dbos architecture much more than temporal since it is much easier to operate on small team. But dbos doesn't have opensource release of web ui, which is most critical part for a workflow management tool. Since competitor have all of itself opensource I don't think dbos will have a chance.

We'll be releasing self-hostable Conductor (the web UI) soon--stay tuned!

looking forward to it!
Post reply on HN