Ask HN: What are best patterns for events in distributed transactions? • Hacker News Board

Re: Ask HN: What are best patterns for events in distributed transactions?

#11

Post by nitwit005 » Fri, Dec 30, 2022, 3:58 AM UTC

The boring question people tend to overlook is "How do I do database backup and restore?".

If you did everything with distributed transactions, and had a single data store that supported it, you could have all backups be at the same transaction checkpoint. When you restored, everything would be in a consistent state.

If everyone is managing their own data in different data stores, and just listening to Kafka or similar, you have to deal with the inconsistent state after a DB restore somehow.

Re: Ask HN: What are best patterns for events in distributed transactions?

#12

Post by mountainreason » Sat, Dec 31, 2022, 3:42 PM UTC

I dont have an answer, I'm just curious about something. I thought that the point of an outbox was that it was local, and could therefore be updated atomically along with any local db changes. What would happen if the orchestrator was unable to update the remote shared cache outbox after a local exception/rolled back transaction?

Re: Ask HN: What are best patterns for events in distributed transactions?

#13

Post by olivermrbl » Mon, Jan 02, 2023, 9:37 AM UTC

mountainreason wrote: post #12

I dont have an answer, I'm just curious about something. I thought that the point of an outbox was that it was local, and could therefore be updated atomically along with any local db changes. What would happen if the orchestrator was unable to update the remote shared cache outbox after a local exception/rolled back transaction?

> What would happen if the orchestrator was unable to update the remote shared cache outbox after a local exception/rolled back transaction

To be sure, I understand your question, you are referring to the following scenario:

- Start global transaction

- Create shared cache entry

- Add events to the shared cache

- Local transaction fails

- Orchestrators starts rolling back local transaction

- Attempts to delete shared cache entry To ensure we don't have stale events in the cache, I would add a TTL, which is configurable for each job (such that you can have long-living events). So in case the orchestrator fails to delete the cache entry, they would be invalidated at some point by exceeding its TTL.