Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Webhooks do’s and dont’s: what we learned after integrating APIs

restful.io

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#71

Post by rakoo »

bazizbaziz wrote: post #16

How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…

Yeah, I feel the best way is just for providers to give a RSS feed as the primary way of listing events and then notify with PubSubHubbub directly. Big advantage: everything already exists and is standard.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#72

Post by stevekemp »

blairanderson wrote: post #69
bazizbaziz wrote: post #16

How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…

I have recently moved all received webhooks to a job queue and have been very happy. you can retry the processing on your own terms.

Agreed.

It also allows you to do testing by injecting pre-cooked payloads into your queue system.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#73

Post by dajonker »

developer2 wrote: post #65
iddqd wrote: post #27

Earlier quoted context omitted.

Maybe webhook providers could provide an endpoint where one could poll for events that failed to deliver.

The good APIs do, but it's still at a loss to both sides. a) The producer of the events has to store them in semi-permanent storage. I've been there and done that - failed webhooks result in a table of tens of millions of rows, even if the memory on each event is only 48 hours. It's astounding how many events fail to process. And I've been through extensive verification that there is truly no problem on our side - it…

You could also help customers who apparently have trouble properly connecting to your APIs by giving better error returns (got type A, expected type B), providing client libraries or giving more extensive support (for a price). Blaming the customer is easy, providing a way for even those "incompetent developers" to interface with you in a way that is easy to understand and debug for all parties is hard.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#74

Post by _pmf_ »

developer2 wrote: post #59
bazizbaziz wrote: post #16

How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…

I would prefer to implement the sending of webhooks in bulk - if the consumer falls behind, they receive up to 100-1000 webhooks per request (depending on the size and complexity of each individual webhook - ids only is 1000, complex documents 100). This drastically cuts down on the number of concurrent requests to a single client when load is high, or the consumer broke down for a period of time. Unfortunately, deve…

Transactions are obviously too enterprisey for fast moving unicorns; better spend 3 weeks to badly hack together a ridiculous farce.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#75

Post by MrBuddyCasino »

sly010 wrote: post #22

Aside: Webhooks are always a pain. Implementing polling is easier for both sides. I routinely have to integrate with random 3rd party systems, some with no or broken webhooks, some with no API at all. It turns out for my customers (this may not be always the case) eventual consistency is more important than timelyness. What I do now every time I need to sync data from a third party is I always implement some sort of…

Thats what I did, too. Poll, fetch and remember and retry errors, and if possible implement a sliding window as a poor man's cursor using dateFrom / dateTo if available.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#77

Post by z3t4 »

bazizbaziz wrote: post #16

How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…

The easiest implementation would be a serial number. Then the client can check for holes in the number series.

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#78

Post by Moru »

dajonker wrote: post #73
developer2 wrote: post #65

Earlier quoted context omitted.

The good APIs do, but it's still at a loss to both sides. a) The producer of the events has to store them in semi-permanent storage. I've been there and done that - failed webhooks result in a table of tens of millions of rows, even if the memory on each event is only 48 hours. It's astounding how many events fail to process. And I've been through extensive verification that there is truly no problem on our side - it…

You could also help customers who apparently have trouble properly connecting to your APIs by giving better error returns (got type A, expected type B), providing client libraries or giving more extensive support (for a price). Blaming the customer is easy, providing a way for even those "incompetent developers" to interface with you in a way that is easy to understand and debug for all parties is hard.

The truly great developers find a better way than only retrying webhooks and prepare a client library that the customer can just plug in to their code :-)

Re: Webhooks do’s and dont’s: what we learned after integrating APIs

#79

Post by notgood »

developer2 wrote: post #59
bazizbaziz wrote: post #16

How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…

I would prefer to implement the sending of webhooks in bulk - if the consumer falls behind, they receive up to 100-1000 webhooks per request (depending on the size and complexity of each individual webhook - ids only is 1000, complex documents 100). This drastically cuts down on the number of concurrent requests to a single client when load is high, or the consumer broke down for a period of time. Unfortunately, deve…

I don't understand, if it's such a nightmare why don't you (the producer) create the code/libraries to use those webhooks? At least in the 2 most common platforms (e.g. PHP, Java)
Post reply on HN