11–20 of 112 posts
Why hasn't Google implemented this yet? Their current solution isn't good enough ( https://developers.google.com/webmasters/ajax-crawling/ ).
Wouldn't the pre-render based on useragent be penalized because Google doesn't like being shown pages differently than non-Googlebot useragents?
Google doesn't like it when they are shown different content than a browsing user. This is roughly the equivalent of pointing Google Agent to a copy of the page requested that happens to be in Memcached instead of spinning up the full app stack to do the render.
This is exactly correct. Regardless of your motivations.
Why hasn't Google implemented this yet? Their current solution isn't good enough ( https://developers.google.com/webmasters/ajax-crawling/ ).
web apps today are so much more than ajax. You have to actually a full blown DOM tree to get what real user-agents renders
Google is less important (they already execute JS), however it's good for sites like Facebook (which doesn't when you share a link).
This looks similar to Meteor's "spiderable" package http://docs.meteor.com/#spiderable
This will get you penalized for having a website that takes forever to load. This is what happens: Googlebot requests page -> your webapp detects googlebot -> you call remote service and request that they crawl your website -> they request the page from you -> you return the regular page, with js that modifies it's look and feel -> the remote service returns the final html and css to your webapp -> your webapp return…
Looking at the prerender's source I did't see any caching mechanism.
What kind of load times have you see rendering your apps?
Have there been recent significant improvements in phantomjs's performance?
This will get you penalized for having a website that takes forever to load. This is what happens: Googlebot requests page -> your webapp detects googlebot -> you call remote service and request that they crawl your website -> they request the page from you -> you return the regular page, with js that modifies it's look and feel -> the remote service returns the final html and css to your webapp -> your webapp return…
This will get you penalized for having a website that takes forever to load. This is what happens: Googlebot requests page -> your webapp detects googlebot -> you call remote service and request that they crawl your website -> they request the page from you -> you return the regular page, with js that modifies it's look and feel -> the remote service returns the final html and css to your webapp -> your webapp return…
Pretty sure the load time problem can be mitigated by caching.
Unless this works w/o adding network roundtrips on each request, it's not a great idea.