21–27 of 27 posts
You can also poll an /events endpoint to get a consistent ordering. I used /events to apply writes from Stripe to a local database for this reason in the tdog CLI: https://table.dog/blog/principles/events-are-better/
Stripe just so happens to offer both?
Not webhook specific but a couple hours today figuring out that some our service calls to internal services look like they open & are sent & processing, but the target server doesnt even see the request for a full 8s sometimes. The call itself was not thrle problem, the service just hadnt started until a long time after data was all sent.
This is rubbish, we've run with guaranteed webhook ordering for years, so the idea that you can't do is laughable. Timestamps don't solve the issue, and neither do "thin payloads" since the receiver has no idea how long to wait before assuming that the order is certain, and if you have a problem on the sender side it could cause logic errors for all of your clients. Most of these problems are solved if the receiver d…
> In this system, the client would request a range (start time, max count) of events via an HTTP request, and the response would include the "end time" that can be used in the next query. What happens if two transactions commit out of order? tx1 with a lower timestamp commits after tx2 with a higher timestamp has committed - and your client just saw tx2's timestamp. Or if you have ≥$maxCount number of events changed…
If two transactions are non-causal, it doesn't matter which order the events arrive in the queue, but once the message is in the queue, the order is fixed.
> Or if you have ≥$maxCount number of events changed the same exact timestamp?
Use a sufficiently precise timestamp that this doesn't happen, or add a counter in the low bits. The only reason to use a timestamp rather than a simple incrementing counter is to make it more convenient for recipients to re-request historical events (eg. I want to replay all events since yesterday) and to make debugging easier, since with a counter it's a bit meaningless.
The timestamp is not meaningful for the actual event, its only purpose is to specify where this event sits in the total order.
P.S. Svix is great, super happy customer here :)
This is rubbish, we've run with guaranteed webhook ordering for years, so the idea that you can't do is laughable. Timestamps don't solve the issue, and neither do "thin payloads" since the receiver has no idea how long to wait before assuming that the order is certain, and if you have a problem on the sender side it could cause logic errors for all of your clients. Most of these problems are solved if the receiver d…
Author here. You obviously CAN guarantee ordering, it's just that you can't guarantee it as the sender, you need cooperation from the receiver. Additionally, putting them in a receive queue on the receiver doesn't solve the issue unless the receiver takes extra care to also read from the queue in strict (non-overlapping) order which is rarely the case, and even then has significant throughput implications. So it real…
Earlier quoted context omitted.
Author here. You obviously CAN guarantee ordering, it's just that you can't guarantee it as the sender, you need cooperation from the receiver. Additionally, putting them in a receive queue on the receiver doesn't solve the issue unless the receiver takes extra care to also read from the queue in strict (non-overlapping) order which is rarely the case, and even then has significant throughput implications. So it real…
Thanks for this, your post resonated with me. It’s good to know that most of what you did is what I ended up doing for a customer implementation (we’re using Odoo and Queue Job to bring in sales from Shopify) and Shopify doesn’t always always guarantee the ordering of their order webhooks payloads.
Really enjoyed the post Tom. I really like the structure of introducing a potential solution then pointing out the problems with it. P.S. Svix is great, super happy customer here :)