Live data from Hacker News

REST Hooks - Stop the polling madness

resthooks.org

71–80 of 117 posts

Re: REST Hooks - Stop the polling madness

#71
post #39
post #32

Earlier quoted context omitted.

Better. That's practically the only thing that needs to be on the home page. You also need to explain who you are and why you're doing this, too. Right now it looks like you have an interest in selling something but you never explain what it is. It reminds me a lot of those sites which sell some miracle product, and are deliberately very long and vague, and it ends finally with some button to order an ebook or someth…

Good point. For us (Zapier), we just waste a lot of time and resources polling. We talked to some of the SaaS services we integrate with and it is the same on their side as well. That is our motivation.

Great - if you're just up front about that I would read this.

Then again if I'm not the audience, well, I guess you know what you're doing.

Re: REST Hooks - Stop the polling madness

#72
Why is this a thing? Your API interfaces with a message queue; your end client uses all the libraries and patterns built up over the years to consume from the queue. That doesn't sound all startup-y like "REST Hooks", but the pattern has been around for a long time.

Re: REST Hooks - Stop the polling madness

#73

> On average, 98.5% of polls are wasted Erm. Is there a source for that?

A bit further down the page, they cite Zapier "Over a representative time period, Zapier polled for changes 30 million times but only took action on 460,000 of those polls (1.5% efficient)."

It's a Zapier project, so they're citing themselves.

https://github.com/zapier/resthooks

Re: REST Hooks - Stop the polling madness

#74

> REST Hooks itself is not a specification, it is a collection of patterns that treat webhooks like subscriptions. Bummer. No matter if you like it or not, a collection of patterns with a name _is_ a specification, just possibly a poorly defined one. See the confusion in this thread. If it were a link to a spec, nobody would be confused. > Skip the pedantic arguments about standards and implementation details This re…

GitHub implements the PSHB subscription API. It's fine, but doesn't really fit in well with the rest of our JSON API. For instance, parameters are sent over as form encoded values like "hub.mode".

http://developer.github.com/v3/repos/hooks/#pubsubhubbub

We also have a regular JSON endpoint for our hooks resource (which is essentially a subscription API).

Re: REST Hooks - Stop the polling madness

#75
post #43
post #42

Earlier quoted context omitted.

Still scratching my head on the Ruby "reference" implementation. When I subscribe resource_subscriptions, who is receiving the update notification? Is the client side js code getting a push? Is the resource_subscriptions a long-polling endpoint? Not very clear at all.

The server itself will just POST a payload to a defined URL. I'm not wildly familiar with the internals of that app or Rails in general, otherwise I'd give more detail!

So all this is just about the following pattern?

A wants to get updates of B. Instead of polling B for changes, it sends B or C an URL of A and says: POST to this URL the moment something changes on B.

If that's the case, this really gets lost on all the hooks/subscriptions/load-mumbojumbo on this site.

Re: REST Hooks - Stop the polling madness

#76
The way the info is presented bugs me. The first "case study" is Zapier. Later, in the "Learn how..." set of links, Zapier is listed as one of many. Look to the footer, and the Github link, and it's of course evident it's a Zapier project.

This project (er hmm, "initiative") is core to Zapier's business. If every service out there had a hook atop their service, it'd make things a lot easier for Zapier. That's cool. What bugs me is the feeling that Zapier's branding in the whole thing is less than transparent.

It seems that open source projects are getting more and more marketing driven, and the way this "initiative" is packaged is a sign of things to come.

Re: REST Hooks - Stop the polling madness

#77
post #8

Am I the only one who has trouble sifting through all this marketing? Even "read more" link leads to an article where half of the text promises what will be covered and another half talks about some subscriptions and how they are managed (they might have been mentioned in the first half, I agree - and I also read text only twice). If someone has a better understanding of how this works, I would appreciate a TL;DR ver…

Yeah, it's pretty bad. I think it comes down to "assuming WebHooks, a suggested 'pattern' to set up the subscription programmatically".

Re: REST Hooks - Stop the polling madness

#78
If for some reason you can't or don't want to implement webhooks, at least make sure you the GET endpoint for any object has a query param that supports fetching the most recently updated or created objects and supports pagination.

It sounds trivial, but you'd be surprised how many APIs don't support one or both of those features. When you're writing an API it might seem unnecessary to start (after all, who could ever have 1000s of ?), but if someone ends up polling your API frequently, having those two features can reduce a lot of unnecessary load for both you and the poller. And, of course, make sure you have an index on the created and/or updated dimensions.

That said, webhooks are terrific. Few things to consider when implementing them:

- Think carefully about the payload you send to the webhook. It's usually a good idea to send some related objects/data because many times when someone gets a webhook payload, that'll trigger calls to your API to get related information you could've reasonably sent them in the initial payload.

- You'll likely want to some way to keep track of errors so if an endpoint starts returning 404s or 500s you have a way to programmatically discard it after X failed attempts.

- In your docs, give sample, "real world" payloads developers can test against. It saves times over creating a RequestBin, pushing there, copying, cURLing, etc. (Remember, you can't set up a webhook to localhost.)

- A nice to have is some sort of retry capability with an exponential back-off. Servers go offline and if they get pushed webhook data then, those messages are lost. You could say, "tough, it's the consumer's responsibility," but if having all the data is important, most people will resort to polling. (Somewhat related, you'd be surprised how often the APIs of some larger SaaS companies are "offline" -- e.g. returning 503 --, so these things do happen.)

Re: REST Hooks - Stop the polling madness

#79
While the paradigm of having a webhook makes more sense, I personally hate working with them. I've yet to see a graceful way of handling local development that doesn't involve localtunnel (or some alternative) and constantly having to un-register and re-register endpoints every time localtunnel gives you a new hostname. I understand the point and would prefer working with them if there was just a good way to handle local development.

Re: REST Hooks - Stop the polling madness

#80
post #79

While the paradigm of having a webhook makes more sense, I personally hate working with them. I've yet to see a graceful way of handling local development that doesn't involve localtunnel (or some alternative) and constantly having to un-register and re-register endpoints every time localtunnel gives you a new hostname. I understand the point and would prefer working with them if there was just a good way to handle l…

This is a great point, is there a better way to do local dev with webhooks? I always used localtunnel which is great it not perfect.
Post reply on HN