Live data from Hacker News

Google's Indexing Javascript more than we thought

distilled.net

1–10 of 35 posts

Re: Google's Indexing Javascript more than we thought

#3

i wonder if it also means we don't need to implement _escaped_fragment_ anymore http://code.google.com/web/ajaxcrawling/docs/getting-started...

Just because GoogleBot can crawl and execute/index javascript, doesn't mean that it will on your site. The best bet would be to keep them. Or take them off and see what happens. If you don't see negative effects, then you will have discovered something interesting.

Re: Google's Indexing Javascript more than we thought

#4

i wonder if it also means we don't need to implement _escaped_fragment_ anymore http://code.google.com/web/ajaxcrawling/docs/getting-started...

Just because GoogleBot can crawl and execute/index javascript, doesn't mean that it will on your site. The best bet would be to keep them. Or take them off and see what happens. If you don't see negative effects, then you will have discovered something interesting.

Yeah I'd definitely say we should continue to follow our established best practices until G gets better at this but Josh's evidence and our continued testing on this subject is very compelling.

Re: Google's Indexing Javascript more than we thought

#5
It occurs to me that if GoogleBot is executing client javascript you could take advantage of Google's resources for computational tasks.

For instance, let me introduce you to SETI@GoogleBot. SETI@GoogleBot is much like SETI@home except it takes advantage of GoogleBot's recently discovered capabilities. Including the SETI@GoogleBot script into your web pages will cause (after the page load event) the page to fetch a chunk of data from the SETI servers via ajax request and proceed to process that data in JavaScript. Eventually, once the data has been processed it will be posted back to the SETI servers (via another ajax request) and then repeat the cycle. Thus enabling you, for the small cost of a page load, have GoogleBot process your SETI data and enhance your SETI@home score.

Obviously, this isn't a new idea (using page loads to process data via JavaScript) but it is an interesting application to exploit GoogleBot's likely vast resources.

Re: Google's Indexing Javascript more than we thought

#6

Earlier quoted context omitted.

Just because GoogleBot can crawl and execute/index javascript, doesn't mean that it will on your site. The best bet would be to keep them. Or take them off and see what happens. If you don't see negative effects, then you will have discovered something interesting.

Yeah I'd definitely say we should continue to follow our established best practices until G gets better at this but Josh's evidence and our continued testing on this subject is very compelling.

I agree on this one... just because Google CAN do it doesn't mean they will.

On that note, I am personally of the belief that the fragments are part of Google's learning/training process for their spiders.

If they sniff the XHR traffic on every domain they encounter a HashBang they can learn lots about the use of AJAX and the types of content being exposed via AJAX.

Re: Google's Indexing Javascript more than we thought

#7
Is there any example of a site having their dynamically generated* disqus comments indexed by google? Disqus is probably one of the most common form of ajax-generated content on the web, so if this were the case that googlebot was actively indexing dynamic content like this, I would expect to see disqus supported.

* disqus has an API to allow you to display disqus comments serverside, so some disqus implementations - think mashable is one - will have comments indexed without the aid of Javascript.

Re: Google's Indexing Javascript more than we thought

#8

Earlier quoted context omitted.

Just because GoogleBot can crawl and execute/index javascript, doesn't mean that it will on your site. The best bet would be to keep them. Or take them off and see what happens. If you don't see negative effects, then you will have discovered something interesting.

Yeah I'd definitely say we should continue to follow our established best practices until G gets better at this but Josh's evidence and our continued testing on this subject is very compelling.

I'd argue that best practice in web development is not requiring JavaScript to load a page, but I'm sure that issue has been done to death in the past.

Re: Google's Indexing Javascript more than we thought

#10
> My personal favorite example of this is Google Translate, which is one of the most accurate machine translating tools on the planet. Google almost sacked it because it was not profitable, and had it not been for public outcry we may have lost access to this technology altogether.

I kind of missed this "public outcry", when did it happen? And if Google listens to public outcry, why did we lose Google Code Search?

Post reply on HN