Emitting domain events with the outbox pattern
1–9 of 9 posts
Re: Emitting domain events with the outbox pattern
#2tl;dr: Because event processing software doesn't do distributed transactions / 2-PC, put an event queue for each microservice in the database where the microservice puts it events, so you get transactions on your events. Then have a subordinate messaging nanoservice empty the locally transactional microservice event queue into the globally untransactional primary event queue.
Re: Emitting domain events with the outbox pattern
#3I have never worked on a project where the db being “down” was anything less than a catastrophe, as seemingly compared to a regular hiccup in this article.
Re: Emitting domain events with the outbox pattern
#4Also a good explanation of the pattern: https://docs.particular.net/nservicebus/outbox/
Re: Emitting domain events with the outbox pattern
#5I have never worked on a project where the db being “down” was anything less than a catastrophe, as seemingly compared to a regular hiccup in this article.
The article just seemed to me like it does not focus on that error case because it's not interesting to the pattern.
You tell your client that the request failed and that's it, right? Or there's certainly other patterns designed to help avoid hard fails when the DB is down, but they seem orthogonal to this one, use them if they're helpful to your project, skip if not?
Re: Emitting domain events with the outbox pattern
#6> The pattern also puts an additional burden on the database. Therefore, we should consider the potential performance hit on the database
Massive understatement, one of the downsides that we've found with this pattern is the increase increase in binlog size for our database replication topology. One must be very careful with this pattern as to not create downstream issues with replication delay and such.
If taking advantage of this pattern in MySQL land, you should look into filtered replication to avoid this outbox table from being copied to every single replica. https://dev.mysql.com/doc/refman/5.7/en/change-replication-f...
Re: Emitting domain events with the outbox pattern
#7That's my favorite part about using messaging in Vitess. I can emit events in a real transaction. It eliminates an entire class of bugs.
Re: Emitting domain events with the outbox pattern
#8Folks looking to implement the outbox pattern with Apache Kafka may find the support provided for it by Debezium useful: https://debezium.io/documentation/reference/stable/transform....
Debezium is implementing log-based change data capture, i.e. there's no polling on the outbox table.
There's also an extension for Quarkus (https://debezium.io/documentation/reference/stable/integrati...) which makes it very easy to implement the pattern in Java applications.
Disclaimer: I work on Debezium
Re: Emitting domain events with the outbox pattern
#9> But what happens if the subscription record is deleted from the DB right before emitting the event due to a bug?
Isn't this still inside an uncommitted transaction? What sort of bug would lead to an uncommitted record being deleted that was not a major bug in the database?
If the actual concern is that the record is deleted before the event is processed (not before it is sent as the article seems to be talking about), why not put foreign keys between the outbox and the users to either prevent that or make sure that both are deleted simultaneously?