Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Deactivates Web Search API

ajax.googleapis.com

Post reply on HN
Print view View original

1–10 of 132 posts

Re: Google Deactivates Web Search API

#2

Post by woodruffw »

Since I can't post both a title and text:

What's interesting about this is that the API has been officially deprecated since 2010 (and "offline" since September 2014), but today is the first day that it's actually become unavailable.

Edit: Additionally, there are probably going to be big repercussions to the web/average person's browsing experience as a result of this. A massive number of sites (and programs, bots, etc) were using this API because of its simplicity (and absence of registration/authentication).

Re: Google Deactivates Web Search API

#5

Post by martinald »

Slightly OT: Does anyone know how RankScanner works? My guess was this but it still seems to work. Their site seems to suggest they have loads of servers, but seems unlikely, I'd have thought Google would have captached them to hell by now.

Very good product. I'm curious how it works though.

Re: Google Deactivates Web Search API

#6

Post by woodruffw »

martinald wrote: post #5

Slightly OT: Does anyone know how RankScanner works? My guess was this but it still seems to work. Their site seems to suggest they have loads of servers, but seems unlikely, I'd have thought Google would have captached them to hell by now. Very good product. I'm curious how it works though.

I'm not acquainted with RankScanner, but my educated guess is that they use one of Google's (many) authenticated APIs[1] for search/content categorization/ranking analysis.

[1]: https://developers.google.com/custom-search/

Re: Google Deactivates Web Search API

#7

Post by martinald »

woodruffw wrote: post #6
martinald wrote: post #5

Slightly OT: Does anyone know how RankScanner works? My guess was this but it still seems to work. Their site seems to suggest they have loads of servers, but seems unlikely, I'd have thought Google would have captached them to hell by now. Very good product. I'm curious how it works though.

I'm not acquainted with RankScanner, but my educated guess is that they use one of Google's (many) authenticated APIs[1] for search/content categorization/ranking analysis. [1]: https://developers.google.com/custom-search/

Thanks. I was under the search impression that this was only for building your own internal search. do you know how much the pricing is?

Re: Google Deactivates Web Search API

#8

Post by woodruffw »

martinald wrote: post #7
woodruffw wrote: post #6

Earlier quoted context omitted.

I'm not acquainted with RankScanner, but my educated guess is that they use one of Google's (many) authenticated APIs[1] for search/content categorization/ranking analysis. [1]: https://developers.google.com/custom-search/

Thanks. I was under the search impression that this was only for building your own internal search. do you know how much the pricing is?

There are some prices listed here: https://developers.google.com/custom-search/json-api/v1/over...

These seem to be for products in a slightly different domain than plain old search, though.

Re: Google Deactivates Web Search API

#9

Post by Jake232 »

martinald wrote: post #5

Slightly OT: Does anyone know how RankScanner works? My guess was this but it still seems to work. Their site seems to suggest they have loads of servers, but seems unlikely, I'd have thought Google would have captached them to hell by now. Very good product. I'm curious how it works though.

Having built a ranktracker before, and still being good friends with people who run one of the biggest rank trackers on the market, using the google API's simply isn't an option.

Simply put, the API never returned the results in the exact same order as the actual search results did.

The best / most reliable rank trackers did (and still do) simply use proxies to get around the google captchas. I've scraped millions of pages from google over the years, and with enough proxies, the correct proxy delays/timeouts, and other little tricks you pick up along the way; you can actually scrape google pretty easily.

This is especially true with the dropping cost of proxies. I'm obviously an exception considering I run several scraping SaaS and have generally specialized in it for years, but I can he hitting x.com with 30,000 different IP's within the hour.

Post reply on HN