This is one of the best arguments for scraping data, if you can manage it. Generally businesses invest much more time and resources into maintaining an up-to-date, usable website than they would on an API, especially if they don't use a service-oriented architecture internally (ie, consume their own API)
OMG, what? Scraping data for offline processing, but certainly should not be for user-facing traffic. Cosmetic tweaks can blow up the scraper's parser, UI technologies are built on a taller stack and are prone to higher latencies and points of failure, yada yada.
APIs Are Forever, Wait No...They Can Go Away at Any Time
11–18 of 18 posts
Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#12There is rarely a good reason to actively destroy old API - section it off, keep it out of new code, maybe even kill the documentation so new developers don't use it, but it's bad business and downright disrespectful to the third-party developers who enrich your platform.
Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#13Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#14Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#15Clear deprecation policies are great, but I also think that every public API needs to be explicitly versioned. This is commonly achieved by including the version in the URL (api.example.com/v3/whatever), but a suitable header (API-Version: 20120420) would also work. Once published, a specific version of an API should never change in a backward-incompatible way, only become superseded by newer versions. Likewise, you…
It makes some good points, although I'm not sure I entirely agree with the conclusion drawn, since it places all of the responsibility on the authors of the client-side code:
https://secure.designinghypermediaapis.com/nodes/bujxbmhffep...
Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#16Clear deprecation policies are great, but I also think that every public API needs to be explicitly versioned. This is commonly achieved by including the version in the URL (api.example.com/v3/whatever), but a suitable header (API-Version: 20120420) would also work. Once published, a specific version of an API should never change in a backward-incompatible way, only become superseded by newer versions. Likewise, you…
That definitely seems like the common-sense approach. There was an interesting article posted on here recently though, which argued that API versioning is an anti-pattern. It makes some good points, although I'm not sure I entirely agree with the conclusion drawn, since it places all of the responsibility on the authors of the client-side code: https://secure.designinghypermediaapis.com/nodes/bujxbmhffep...
"This can be handled in the opposite way we dealt with new functionality: if you don't see something, don't display it."
In my experience that is just too simplistic idea to cover all cases hence the need for versioning.
Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#17Clear deprecation policies are great, but I also think that every public API needs to be explicitly versioned. This is commonly achieved by including the version in the URL (api.example.com/v3/whatever), but a suitable header (API-Version: 20120420) would also work. Once published, a specific version of an API should never change in a backward-incompatible way, only become superseded by newer versions. Likewise, you…
That definitely seems like the common-sense approach. There was an interesting article posted on here recently though, which argued that API versioning is an anti-pattern. It makes some good points, although I'm not sure I entirely agree with the conclusion drawn, since it places all of the responsibility on the authors of the client-side code: https://secure.designinghypermediaapis.com/nodes/bujxbmhffep...
It's possible to view different versions of an API as different representations of the same data. For example, an HTTP server is permitted to return different responses, or even return an error, depending on the contents of the client's "Accept:" header. If versioning was done with a header instead of the URL, it would achieve essentially the same results. The client would be saying "I want /user/123/posts/recent, and I want it in the APIv3 format."
If an API method changes so much that it is no longer recognizable as a different representation of the same data, the URL should probably change to reflect the new content. The old URL would then return data only when the old version is requested, and "406 Not Acceptable" when the new version is requested. So I don't see any reason why API versioning can't be RESTful. It's just slightly more difficult to do it RESTfully, compared to adding the version in the URL.
In any case, I'd rather take practicality over RESTful purity.
Re: APIs Are Forever, Wait No...They Can Go Away at Any Time
#18This is one of the best arguments for scraping data, if you can manage it. Generally businesses invest much more time and resources into maintaining an up-to-date, usable website than they would on an API, especially if they don't use a service-oriented architecture internally (ie, consume their own API)
What type of business can afford being down every time the site they're scrapping is tweaked?
Plus, building a parser isn't always that much more work than trying to build a client for a convoluted API.
It's obviously case-by-case, but I think a lot of people dismiss scraping outright when there are a lot of useful applications.