Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres transactions are a distributed systems superpower

dbos.dev

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Postgres transactions are a distributed systems superpower

#31

Post by KraftyOne »

belinder wrote: post #21
mrkeen wrote: post #18

I walked away from a job interview a few years ago on this point. One of the technical questions was "if you have a db and a message queue, how do you get your update to alter both or neither (i.e. transactionally)"? I thought about it for a couple of minutes, then came back with something like "I can't, and you can't either." Then I proposed the usual spiel about using a replicated-state-machine/write-ahead-log/even…

Why not just put the message queue in the same db

That's what the post is about! Once you're doing that, you really do have transactions between the state and the queue.

Re: Postgres transactions are a distributed systems superpower

#32

Post by mrkeen »

jayd16 wrote: post #29
mrkeen wrote: post #18

I walked away from a job interview a few years ago on this point. One of the technical questions was "if you have a db and a message queue, how do you get your update to alter both or neither (i.e. transactionally)"? I thought about it for a couple of minutes, then came back with something like "I can't, and you can't either." Then I proposed the usual spiel about using a replicated-state-machine/write-ahead-log/even…

It's a bit of trick that the outbox to queue part of it likely needs to support "at least once but duplicates possible" into the queue.

"Send multiple times from D to Q and deduplicate with a UUID" (idempotency) is well short of "insert into both D and Q or neither" (atomicity)

Re: Postgres transactions are a distributed systems superpower

#33

Post by mrkeen »

belinder wrote: post #21
mrkeen wrote: post #18

I walked away from a job interview a few years ago on this point. One of the technical questions was "if you have a db and a message queue, how do you get your update to alter both or neither (i.e. transactionally)"? I thought about it for a couple of minutes, then came back with something like "I can't, and you can't either." Then I proposed the usual spiel about using a replicated-state-machine/write-ahead-log/even…

Why not just put the message queue in the same db

Step 1: identify that you and at least one other node are separated by distance, and some lossy communication channel, and therefore form a distributed system.

Step 2: propose a source of truth that everyone can listen to. Hearing the same facts in the same order should put everyone in the same state (eventual consistency)

Step 3 (you are here): try to do better than EC, by merging the external queue into one of the nodes, making it the master.

Step 4: Now there's no distance between the nodes, so no need to solve the distributed systems problem and you can retire the queue.

Re: Postgres transactions are a distributed systems superpower

#35

Post by vlovich123 »

mrkeen wrote: post #18

I walked away from a job interview a few years ago on this point. One of the technical questions was "if you have a db and a message queue, how do you get your update to alter both or neither (i.e. transactionally)"? I thought about it for a couple of minutes, then came back with something like "I can't, and you can't either." Then I proposed the usual spiel about using a replicated-state-machine/write-ahead-log/even…

> Sure enough it sounded like this article

FWIW The article literally talks about the challenges with getting this to actually work and recommends removing it and just using the DB for everything.

Re: Postgres transactions are a distributed systems superpower

#36

Post by mrkeen »

evilturnip wrote: post #6

Can you use postgres as a state store for a distributed application? It seems this article is trending toward that view: If you can maintain transactional consistency along with application workflow state, then would this generalize to maintaining distributed application state in general? The follow-up would be: Would this be preferable to Valkey/Redis?

> then would this generalize to maintaining distributed application state in general?

Yes, in the sense of 'too good to be true'

Re: Postgres transactions are a distributed systems superpower

#37

Post by LgWoodenBadger »

KraftyOne wrote: post #8
bsaul wrote: post #3

i don't understand the last point of UDF. Either you need the state to be updated atomically across different systems or you don't. But writing a row in a system in order to update the second one at any random time in the future isn't really much different from enqueuing a job in queue.

The key is that the UDF's enqueue is transactional with the database update. Let's say the database update is inserting a new order. This provides the guarantee that if a new order is inserted, a job to process the order is also enqueued. It's impossible for a new order to be inserted without its processing job also being enqueued. Then the durable workflow/queue system is responsible for making sure the processing j…

And if that job never runs? Or if that job runs and then fails to commit that it ran in postgres?

Re: Postgres transactions are a distributed systems superpower

#38

Post by mrkeen »

munk-a wrote: post #10
tomjakubowski wrote: post #5

Earlier quoted context omitted.

I don't think it's true that distributed and decentralized mean the same thing. A hub and spoke rail system is centralized, but it's still a distributed system, if it has multiple trains running concurrently.* A distributed system has to coordinate somehow, and a single central DB is one way of doing it. *: edit, maybe a better example here is a rail system with a single central dispatcher is centralized but may stil…

In fact - if you're building a very large distributed system the goal is usually to shrink that centralized component to the smallest and most robust surface you can. If the system is well designed it is amazing just how much consistency power you can get from a tiny component of centralization. There are always tradeoffs of course, but building a truly decentralized system requires some really difficult compromises…

> The two general's problem is a great piece of reading on this topic

It is!

And the solution is to add an extra general on the left side. Let's call him Outus Boxus. The two generals on the left side can communicate in perfect lockstep. Then if you need the general on the right to find out about something, you can send a few workers to tell him or something...

More seriously though, you can have a DS for two reasons: tech or political.

Tech means scaling or reliability. So clients can be serviced by any of the nodes.

Political means different actors don't have a central authority. You can't stick two banks into one db.

This technique doesn't seem to address either aspect.

Re: Postgres transactions are a distributed systems superpower

#39

Post by jayd16 »

mrkeen wrote: post #32
jayd16 wrote: post #29

Earlier quoted context omitted.

It's a bit of trick that the outbox to queue part of it likely needs to support "at least once but duplicates possible" into the queue.

"Send multiple times from D to Q and deduplicate with a UUID" (idempotency) is well short of "insert into both D and Q or neither" (atomicity)

What are you saying here? I'm pointing out that you need to be ok with the lack of exactly once transaction between O and Q. Maybe you're agreeing and simply saying that's a fine?

Re: Postgres transactions are a distributed systems superpower

#40

Post by jayd16 »

KraftyOne wrote: post #30
jayd16 wrote: post #29

Earlier quoted context omitted.

It's a bit of trick that the outbox to queue part of it likely needs to support "at least once but duplicates possible" into the queue.

Every item will be written to the queue exactly once (as the update is transactional). Queue processing may need at-least-once semantics, yes, depending on what exactly you're doing.

The queue write is not in the transaction. The proposed trick is that that is ok because an outbox is able to be transacted on. It kicks the can some what...
Post reply on HN