Javascript apps can be fully crawlable

I'm also not understanding the use-case for this project. Everytime the topic of "Web Apps", "JavaScript Apps", "Single page web apps" comes up, evangelists point out that they are applications (or skyscrapers), not just fancy decorators for website content.

So exactly what is this project delivering as fallback content? A server-generated website?

This project just seems pointlessly backwards. Simulating a feature that the JavaScript framework has already deliberately broken. One that introduces a server-side dependency on a project deliberately chosen not to have a server-side framework.

This just looks like a waste of effort, when building the JavaScript application properly the first time, with progressive enhancement, covers this exact use-case, and far, far more use-cases.

The time would have been better spent fixing these evidently broken JavaScript frameworks - Angular, ember, Backbone. Or at least to fix the tutorial documentation to explain how to build Web things properly. (This stuff isn't difficult, it just requires discipline)

I call hokum on people saying there's a difference between Websites and Web apps (or the plethora of terms used to obfuscate that: Single-page apps, JavaScript apps). This project proves that these are just Websites, built improperly, and this is the fudge that tries to repair that for Googlebot.

>Javascript apps can be fully crawlable yes, and i think it's cool that you try to provide a solution as a service for this.

but as with every technology, there are some tradeoffs

a) serving google a different response bases on the user-agent is the definition of cloaking (it's not misleading or malicious cloaking, it's cloaking non the less)

b) you hardcode a dependency to a third party server - you have no control over - into your app (and from the sample code on the page, there is no fallback available if this server is down)

c) there are latency/web-performance issue i.e.: for a first time request by a search engine the roundtrip would look like so:

[googlebot GET for page -> googlebot detected -> app GET to prerender.io -> prerender.io GET to page -> app delivers page -> prerender.io returns page to app -> app returns page to googlebot]

this will always be slower than

[googlebot GET for page -> app returns page to googlebot]

so basically the prerender.io approach creates some issues. said that. we don't have - yet - another "no tread-off" solution

the "make ajax crawlable" approach basically allows - non malicious, non misleading - cloaking https://developers.google.com/webmasters/ajax-crawling/docs/...

(sorry google, but ?_escaped_fragment_= was really one of your must stupidest specs ever, even worse then "nofollow")

so if you target "?_escaped_fragment_=" in the GET request, and not the user-agent cloaking a.k.a. sending different responses is ok

but: it creates a double googlebot crawl issue i.e.:

[googlebot GET http://www.exmaple.com/test -> googlebot parses HTML and finds in the HTML -> googlebot pushes http://www.exmaple.com/test?_escaped_fragment_= into its "stuff to crawl-queue" (a.k.a. discovery-queue) -> googlebot crawls http://www.exmaple.com/test?_escaped_fragemten_= -> gets server side get request (or if you would use a prerender.io service the whole roundtrip to the prerender.io site would start) ]

this is a no go if you have a big site with hundred of thousands to millions of pages.

and there is another much, much bigger issue:

  * showing JS clients 
  * and "other only-partially-JS clients" (google parses some JS) different responses 

why? if there is no direct feedback, then there is no direct feedback!

non-responsive mobile site currently offer overall poor user experience, why? because all the guys working on the site sit in front of their fat office desktops. no feedback equals crap in the long run.

and it's worse for "for robots only" views, because people just don't have to live with the crap they server spits out, as they always just see the fancy JS versions. since the hashbang ajax crawl-able spec came out it consulted some clients on this question, everyone who choose the _escaped_fragment_ road anyway did regret it later on. even if the the first iteration works, 1000 roll out later, it doesn't - if there is no direct feedback, then there is no direct feedback.

conclusion: if you have a bit site and want to do big-scale (lots of pages) SEO you are stuck with landingpages and delivering HTML + content via the server + progressive enhancement for functionality, until the day google get's its act together.

and for first-view webperformance i recommend the progressive enhancement approach anyway, too.

https://github.com/icodeforlove/node-express-renderer

It uses phantomjs but removes all the styles initially so the rendering time is much faster. (my ember app was averaging 70MS to render, but i prefetch the page data)