Apostroph'ed.
Webhooks do’s and dont’s: what we learned after integrating APIs
41–50 of 83 posts
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#42How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it. Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems? Seems to me like the best way to a…
By periodic reconciliation of the full dataset.
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#43This is going to sound bizarre, but why do webhooks and not just an AMQP queue? I get that receiving HTTP POSTs is easier, but it just seems better to setup a publisher/subscriber relationship. That way, if a subscriber goes down, they can always catch up. And publishers can allow messages to sit in the queue with a TTL and max_size. It seems like a win-win for everyone.
It's a whole lot simpler to do secure cross-organization HTTP requests than it is to figure out how to have multiple AMQP subscribers from untrusted companies.
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#44Earlier quoted context omitted.
Stripe has a retry policy as well. You can set up something where it will alert you if there are too many failures in a certain time period. That isn't offered by Stripe but you can build it. If you mean in the case of "catastrophic failure", there is none. If there is a "catastrophic failure" (machine gets shut off for a week, data center blown up, whatever), there are probably bigger issues or we probably would alr…
Stripe has an "events" API that can be polled to receive the same content that you would have received via Webhook [1]. (Disclaimer: I work there.) If you missed some Webhooks due to an application failure, it's possible to page through it and look for omissions. I've spoken to at least one person integrating who had this sort of setup running as a regular process to protect against the possibility of dropped Webhook…
Oh gosh, that is super neat! :)
I never even thought of using it that way. I just use events to check that it is a valid Stripe event (Probably easier / better to set up the ELB to only listen to certain addresses)
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#45Also, in your documentation, please show what the webhook events will look like since developers actually want to write code and not guess at what we will get. cough Stripe. ( https://stripe.com/docs/api#events )
The implication was meant to be that the information under `data/object` is simply a full representation of another API resource of the type on which the event occurred, and that you can look elsewhere in the documentation to see exactly what each type will look like (you can see a subscription embedded in the sample response for example). Fair enough that we could rewrite this to be more explicit about that though!…
This is what I love. I purposefully talk about Stripe on here just because I know Stripe people browse HN.
Stripe is great though. :)
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#46I think the "securing webhooks" section is missing some critical tips that we've learned in production. 1) Resolve the DNS of the webhook URL, and compare all returned addresses from that resolution against an IP blacklist, which includes all RFC1918 addresses, EC2 instance metadata, and any other concerning addresses. 2) Even though it seems like you'd want to, do NOT blindly return an unexpected response to the per…
> IP blacklist Please stop trying to enumerate badness. This is and always will be incomplete. http://www.ranum.com/security/computer_security/editorials/d...
Ehh. I disagree with both Default Permit and Enumerating Badness--I think they have their place. If I run a club do I background check and whitelist every customer? Or to a blacklist the troublemakers? The problems cited in the article were reasonable decisions at the time, but years later grew into headaches when the use-cases changed.
Does their no Default Permit policy apply to network egress? Do I have to approve each and every application that wants to connect to the Internet? I think the leaving port 80 open because it was whitelisted is why so many things tunnel through port 80 instead of using other protocols and ports. Now how do you filter and whitelist traffic?
His example of antivirus products using Enumerating Badness is a market failing more than anything else. I'm not sure I see the alternative for a naive user. Call a specialist to investigate their use-cases and "open the system" to accommodate? Any time you want to update your tool or workflow or try something new have that specialist come out and reevaluate your system?
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#47It would have been interesting to read about tests for web-hooks. How do you do integration testing for example?
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#48I'd like to follow up on the statement that the OpenAPI tools do not support webhooks. This is slated to change in an upcoming version of the OpenAPI-specification. Check out https://github.com/OAI/OpenAPI-Specification/pull/763 to see details. As soon as this is released, it only be a matter of time before Swagger and the rest support webhooks.
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#49So "push technology" is called "webhooks" now? How does this all integrate with HTTP 2? Can you get your notifications over a channel you already have open for other reasons?
Not quite. "Push technology" is sort of an all-encompassing term for server-to-client updates whereas webhooks pertain specifically to HTTP callbacks. An example of a webhook would be GitHub making a POST request to some URL (set by the user) whenever new commits are made to a repo. Push technology might take the form of webhooks, long polling, WebSockets etc.
Webhooks are traditional HTTP requests so I don't believe HTTP/2 changes anything. The ability to differentiate notifications depends on the service / API you're integrating with.
Re: Webhooks do’s and dont’s: what we learned after integrating APIs
#50It's also really handy when API providers give a nice webhook UI that lets you view and resend webhooks during development.
BitBucket is a very good example of web hook integrations done right. Relatable, logged, and well documented. I learned from their UI when I implemented my own version.